Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1440593 > unrolled thread

Re: [PATCH 1/1] arm64/hugetlb: clear PG_dcache_clean if the page is dirty when munmap

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCH 1/1] arm64/hugetlb: clear PG_dcache_clean if the page is  dirty when munmap "Leizhen (ThunderTown)" <thunder.leizhen@huawei.com> - 2016-07-11 14:50 +0200
    Re: [PATCH 1/1] arm64/hugetlb: clear PG_dcache_clean if the page is  dirty when munmap Catalin Marinas <catalin.marinas@arm.com> - 2016-07-12 17:40 +0200
      Re: [PATCH 1/1] arm64/hugetlb: clear PG_dcache_clean if the page is  dirty when munmap "Leizhen (ThunderTown)" <thunder.leizhen@huawei.com> - 2016-07-20 04:50 +0200
        Re: [PATCH 1/1] arm64/hugetlb: clear PG_dcache_clean if the page is  dirty when munmap Catalin Marinas <catalin.marinas@arm.com> - 2016-07-20 11:20 +0200

#1440593 — Re: [PATCH 1/1] arm64/hugetlb: clear PG_dcache_clean if the page is dirty when munmap

On 2016/7/9 0:13, Catalin Marinas wrote:
> On Fri, Jul 08, 2016 at 11:24:26PM +0800, Leizhen (ThunderTown) wrote:
>> On 2016/7/8 21:54, Catalin Marinas wrote:
>>> On Fri, Jul 08, 2016 at 11:36:57AM +0800, Leizhen (ThunderTown) wrote:
>>>> On 2016/7/7 23:37, Catalin Marinas wrote:
>>>>> On Thu, Jul 07, 2016 at 08:09:04PM +0800, Zhen Lei wrote:
>>>>>> At present, PG_dcache_clean is only cleared when the related huge page
>>>>>> is about to be freed. But sometimes, there maybe a process is in charge
>>>>>> to copy binary codes into a shared memory, and notifies other processes
>>>>>> to execute base on that. For the first time, there is no problem, because
>>>>>> the default value of page->flags is PG_dcache_clean cleared. So the cache
>>>>>> will be maintained at the time of set_pte_at for other processes. But if
>>>>>> the content of the shared memory have been updated again, there is no
>>>>>> cache operations, because the PG_dcache_clean is still set.
>>>>>>
>>>>>> For example:
>>>>>> Process A
>>>>>> 	open a hugetlbfs file
>>>>>> 	mmap it as a shared memory
>>>>>> 	copy some binary codes into it
>>>>>> 	munmap
>>>>>>
>>>>>> Process B
>>>>>> 	open the hugetlbfs file
>>>>>> 	mmap it as a shared memory, executable
>>>>>> 	invoke the functions in the shared memory
>>>>>> 	munmap
>>>>>>
>>>>>> repeat the above steps.
>>>>>
>>>>> Does this work as you would expect with small pages (and for example
>>>>> shared file mmap)? I don't want to have a different behaviour between
>>>>> small and huge pages.
>>>>
>>>> The small pages also have this problem, I will try to fix it too.
> [...]
>>> If both cases need solving, we might better move the fix in the
>>> __sync_icache_dcache() function. Untested:
>>
>> At first I also want to fix it as below. But I'm not sure which time the PageDirty
>> will be cleared, and if two or more processes mmap it as executable, cache operations
>> will be duplicated. At present, I really have not found any good place to clear
>> PG_dcache_clean. So the below modification may be the best choice, concisely and clearly.
>>
>>> ------------8<----------------
>>> diff --git a/arch/arm64/mm/flush.c b/arch/arm64/mm/flush.c
>>> index dbd12ea8ce68..c753fa804165 100644
>>> --- a/arch/arm64/mm/flush.c
>>> +++ b/arch/arm64/mm/flush.c
>>> @@ -75,7 +75,8 @@ void __sync_icache_dcache(pte_t pte, unsigned long addr)
>>>  	if (!page_mapping(page))
>>>  		return;
>>>  
>>> -	if (!test_and_set_bit(PG_dcache_clean, &page->flags))
>>> +	if (!test_and_set_bit(PG_dcache_clean, &page->flags) ||
>>> +	    PageDirty(page))
>>>  		sync_icache_aliases(page_address(page),
>>>  				    PAGE_SIZE << compound_order(page));
>>>  	else if (icache_is_aivivt())
>>> ----------------8<---------------------
>>>
>>> BTW, can you make your tests (source) available somewhere?
>>
>> Both cases worked well with this patch.
> 
> Now I'm even more confused ;). IIUC, after an msync() in user space we
> should flush the pages to disk via write_cache_pages(). This function
> calls clear_page_dirty_for_io() after which PageDirty() is no longer
> true. I can't tell how a subsequent mmap() can see the written pages as
> dirty.
> 

As my tracing, both cases invoked empty function.

int vfs_fsync_range(struct file *file, loff_t start, loff_t end, int datasync)
	......
	return file->f_op->fsync(file, start, end, datasync);
}

const struct file_operations hugetlbfs_file_operations = {
	.fsync			= noop_fsync,

static const struct file_operations shmem_file_operations = {
	.mmap		= shmem_mmap,
#ifdef CONFIG_TMPFS
	.fsync		= noop_fsync,

[toc] | [next] | [standalone]

#1441513

On Mon, Jul 11, 2016 at 08:43:32PM +0800, Leizhen (ThunderTown) wrote:
> On 2016/7/9 0:13, Catalin Marinas wrote:
> > On Fri, Jul 08, 2016 at 11:24:26PM +0800, Leizhen (ThunderTown) wrote:
> >> On 2016/7/8 21:54, Catalin Marinas wrote:
> >>> ------------8<----------------
> >>> diff --git a/arch/arm64/mm/flush.c b/arch/arm64/mm/flush.c
> >>> index dbd12ea8ce68..c753fa804165 100644
> >>> --- a/arch/arm64/mm/flush.c
> >>> +++ b/arch/arm64/mm/flush.c
> >>> @@ -75,7 +75,8 @@ void __sync_icache_dcache(pte_t pte, unsigned long addr)
> >>>  	if (!page_mapping(page))
> >>>  		return;
> >>>  
> >>> -	if (!test_and_set_bit(PG_dcache_clean, &page->flags))
> >>> +	if (!test_and_set_bit(PG_dcache_clean, &page->flags) ||
> >>> +	    PageDirty(page))
> >>>  		sync_icache_aliases(page_address(page),
> >>>  				    PAGE_SIZE << compound_order(page));
> >>>  	else if (icache_is_aivivt())
> >>> ----------------8<---------------------
> >>>
> >>> BTW, can you make your tests (source) available somewhere?
> >>
> >> Both cases worked well with this patch.
> > 
> > Now I'm even more confused ;). IIUC, after an msync() in user space we
> > should flush the pages to disk via write_cache_pages(). This function
> > calls clear_page_dirty_for_io() after which PageDirty() is no longer
> > true. I can't tell how a subsequent mmap() can see the written pages as
> > dirty.
> 
> As my tracing, both cases invoked empty function.
> 
> int vfs_fsync_range(struct file *file, loff_t start, loff_t end, int datasync)
> 	......
> 	return file->f_op->fsync(file, start, end, datasync);
> }
> 
> const struct file_operations hugetlbfs_file_operations = {
> 	.fsync			= noop_fsync,
> 
> static const struct file_operations shmem_file_operations = {
> 	.mmap		= shmem_mmap,
> #ifdef CONFIG_TMPFS
> 	.fsync		= noop_fsync,

I was referring to standard filesystem (e.g. ext4) writes where, IIUC,
the PageDirty() status is cleared after I/O but it's not necessarily
removed from the page cache.

-- 
Catalin

[toc] | [prev] | [next] | [standalone]

#1446901

On 2016/7/12 23:35, Catalin Marinas wrote:
> On Mon, Jul 11, 2016 at 08:43:32PM +0800, Leizhen (ThunderTown) wrote:
>> On 2016/7/9 0:13, Catalin Marinas wrote:
>>> On Fri, Jul 08, 2016 at 11:24:26PM +0800, Leizhen (ThunderTown) wrote:
>>>> On 2016/7/8 21:54, Catalin Marinas wrote:
>>>>> ------------8<----------------
>>>>> diff --git a/arch/arm64/mm/flush.c b/arch/arm64/mm/flush.c
>>>>> index dbd12ea8ce68..c753fa804165 100644
>>>>> --- a/arch/arm64/mm/flush.c
>>>>> +++ b/arch/arm64/mm/flush.c
>>>>> @@ -75,7 +75,8 @@ void __sync_icache_dcache(pte_t pte, unsigned long addr)
>>>>>  	if (!page_mapping(page))
>>>>>  		return;
>>>>>  
>>>>> -	if (!test_and_set_bit(PG_dcache_clean, &page->flags))
>>>>> +	if (!test_and_set_bit(PG_dcache_clean, &page->flags) ||
>>>>> +	    PageDirty(page))
>>>>>  		sync_icache_aliases(page_address(page),
>>>>>  				    PAGE_SIZE << compound_order(page));
>>>>>  	else if (icache_is_aivivt())
>>>>> ----------------8<---------------------
Hi, Catalin:
  Do you plan to send this patch? My colleagues told me that if our patches are quite
different, it should be Signed-off-by you.

  I searched all Linux source code, __sync_icache_dcache is only called by set_pte_at,
and some check conditions(especially pte_exec) will limit its impact.

	if (pte_user(pte) && pte_exec(pte) && !pte_special(pte))
		__sync_icache_dcache(pte, addr);

>>>>>
>>>>> BTW, can you make your tests (source) available somewhere?
>>>>
>>>> Both cases worked well with this patch.
>>>
>>> Now I'm even more confused ;). IIUC, after an msync() in user space we
>>> should flush the pages to disk via write_cache_pages(). This function
>>> calls clear_page_dirty_for_io() after which PageDirty() is no longer
>>> true. I can't tell how a subsequent mmap() can see the written pages as
>>> dirty.
>>
>> As my tracing, both cases invoked empty function.
>>
>> int vfs_fsync_range(struct file *file, loff_t start, loff_t end, int datasync)
>> 	......
>> 	return file->f_op->fsync(file, start, end, datasync);
>> }
>>
>> const struct file_operations hugetlbfs_file_operations = {
>> 	.fsync			= noop_fsync,
>>
>> static const struct file_operations shmem_file_operations = {
>> 	.mmap		= shmem_mmap,
>> #ifdef CONFIG_TMPFS
>> 	.fsync		= noop_fsync,
> 
> I was referring to standard filesystem (e.g. ext4) writes where, IIUC,
> the PageDirty() status is cleared after I/O but it's not necessarily
> removed from the page cache.
> 

[toc] | [prev] | [next] | [standalone]

#1447119

On Wed, Jul 20, 2016 at 10:46:27AM +0800, Leizhen (ThunderTown) wrote:
> >>>> On 2016/7/8 21:54, Catalin Marinas wrote:
> >>>>> ------------8<----------------
> >>>>> diff --git a/arch/arm64/mm/flush.c b/arch/arm64/mm/flush.c
> >>>>> index dbd12ea8ce68..c753fa804165 100644
> >>>>> --- a/arch/arm64/mm/flush.c
> >>>>> +++ b/arch/arm64/mm/flush.c
> >>>>> @@ -75,7 +75,8 @@ void __sync_icache_dcache(pte_t pte, unsigned long addr)
> >>>>>  	if (!page_mapping(page))
> >>>>>  		return;
> >>>>>  
> >>>>> -	if (!test_and_set_bit(PG_dcache_clean, &page->flags))
> >>>>> +	if (!test_and_set_bit(PG_dcache_clean, &page->flags) ||
> >>>>> +	    PageDirty(page))
> >>>>>  		sync_icache_aliases(page_address(page),
> >>>>>  				    PAGE_SIZE << compound_order(page));
> >>>>>  	else if (icache_is_aivivt())
> >>>>> ----------------8<---------------------
> 
> Do you plan to send this patch? My colleagues told me that if our
> patches are quite different, it should be Signed-off-by you.

The reason I'm not sending it is that I don't fully understand how it
solves the problem for a shared file mmap(), not just hugetlbfs. As I
said in an earlier email: after an msync() in user space we
should flush the pages to disk via write_cache_pages(). This function
calls clear_page_dirty_for_io() after which PageDirty() is no longer
true. I can't tell how a subsequent mmap() can see the written pages as
dirty.

> I searched all Linux source code, __sync_icache_dcache is only called
> by set_pte_at, and some check conditions(especially pte_exec) will
> limit its impact.
> 
> 	if (pte_user(pte) && pte_exec(pte) && !pte_special(pte))
> 		__sync_icache_dcache(pte, addr);

Yes, and set_pte_at() would be called as a result of a page fault when
accessing the mmap'ed file.

-- 
Catalin

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web