Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1682555 > unrolled thread

[RFC PATCH 0/1] mm/mremap: add MREMAP_MIRROR flag

Back to article view | Back to linux.kernel

Contents

  [RFC PATCH 0/1] mm/mremap: add MREMAP_MIRROR flag Mike Kravetz <mike.kravetz@oracle.com> - 2017-07-06 18:20 +0200
    Re: [RFC PATCH 0/1] mm/mremap: add MREMAP_MIRROR flag Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2017-07-07 10:30 +0200
      Re: [RFC PATCH 0/1] mm/mremap: add MREMAP_MIRROR flag Mike Kravetz <mike.kravetz@oracle.com> - 2017-07-07 19:10 +0200
    Re: [RFC PATCH 0/1] mm/mremap: add MREMAP_MIRROR flag Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2017-07-07 13:10 +0200
      Re: [RFC PATCH 0/1] mm/mremap: add MREMAP_MIRROR flag Mike Kravetz <mike.kravetz@oracle.com> - 2017-07-07 19:20 +0200

#1682555 — [RFC PATCH 0/1] mm/mremap: add MREMAP_MIRROR flag

The mremap system call has the ability to 'mirror' parts of an existing
mapping.  To do so, it creates a new mapping that maps the same pages as
the original mapping, just at a different virtual address.  This
functionality has existed since at least the 2.6 kernel [1].  A comment
was added to the code to help preserve this feature.

The Oracle JVM team has discovered this feature and used it while
prototyping a new garbage collection model.  This new model shows promise,
and they are considering its use in a future release.  However, since
the only mention of this functionality is a single comment in the kernel,
they are concerned about its future.

I propose the addition of a new MREMAP_MIRROR flag to explicitly request
this functionality.  The flag simply provides the same functionality as
the existing undocumented 'old_size == 0' interface.  As an alternative,
we could simply document the 'old_size == 0' interface in the man page.
In either case, man page modifications would be needed.

Future Direction

After more formally adding this to the API (either new flag or documenting
existing interface), the mremap code could be enhanced to optimize this
case.  Currently, 'mirroring' only sets up the new mapping.  It does not
create page table entries for new mapping.  This could be added as an
enhancement.

The JVM today has the option of using (static) huge pages.  The mremap
system call does not fully support huge page mappings today.  You can
use mremap to shrink the size of a huge page mapping, but it can not be
used to expand or mirror a mapping.  Such support is fairly straight
forward.

[1] https://lkml.org/lkml/2004/1/12/260

Mike Kravetz (1):
  mm/mremap: add MREMAP_MIRROR flag for existing mirroring functionality

 include/uapi/linux/mman.h       |  5 +++--
 mm/mremap.c                     | 23 ++++++++++++++++-------
 tools/include/uapi/linux/mman.h |  5 +++--
 3 files changed, 22 insertions(+), 11 deletions(-)

-- 
2.7.5

[toc] | [next] | [standalone]

#1683015

On 07/06/2017 09:47 PM, Mike Kravetz wrote:
> The mremap system call has the ability to 'mirror' parts of an existing
> mapping.  To do so, it creates a new mapping that maps the same pages as
> the original mapping, just at a different virtual address.  This
> functionality has existed since at least the 2.6 kernel [1].  A comment
> was added to the code to help preserve this feature.


Is this the comment ? If yes, then its not very clear.

	/*
	 * We allow a zero old-len as a special case
	 * for DOS-emu "duplicate shm area" thing. But
	 * a zero new-len is nonsensical.
	 */


> 
> The Oracle JVM team has discovered this feature and used it while
> prototyping a new garbage collection model.  This new model shows promise,
> and they are considering its use in a future release.  However, since
> the only mention of this functionality is a single comment in the kernel,
> they are concerned about its future.
> 
> I propose the addition of a new MREMAP_MIRROR flag to explicitly request
> this functionality.  The flag simply provides the same functionality as
> the existing undocumented 'old_size == 0' interface.  As an alternative,
> we could simply document the 'old_size == 0' interface in the man page.
> In either case, man page modifications would be needed.

Right. Adding MREMAP_MIRROR sounds cleaner from application programming
point of view. But it extends the interface.

> 
> Future Direction
> 
> After more formally adding this to the API (either new flag or documenting
> existing interface), the mremap code could be enhanced to optimize this
> case.  Currently, 'mirroring' only sets up the new mapping.  It does not
> create page table entries for new mapping.  This could be added as an
> enhancement.

Then how it achieves mirroring, both the pointers should see the same
data, that can happen with page table entries pointing to same pages,
right ?

[toc] | [prev] | [next] | [standalone]

#1683325

On 07/07/2017 01:19 AM, Anshuman Khandual wrote:
> On 07/06/2017 09:47 PM, Mike Kravetz wrote:
>> The mremap system call has the ability to 'mirror' parts of an existing
>> mapping.  To do so, it creates a new mapping that maps the same pages as
>> the original mapping, just at a different virtual address.  This
>> functionality has existed since at least the 2.6 kernel [1].  A comment
>> was added to the code to help preserve this feature.
> 
> 
> Is this the comment ? If yes, then its not very clear.
> 
> 	/*
> 	 * We allow a zero old-len as a special case
> 	 * for DOS-emu "duplicate shm area" thing. But
> 	 * a zero new-len is nonsensical.
> 	 */
> 

Yes, I believe that is the comment.

>>
>> The Oracle JVM team has discovered this feature and used it while
>> prototyping a new garbage collection model.  This new model shows promise,
>> and they are considering its use in a future release.  However, since
>> the only mention of this functionality is a single comment in the kernel,
>> they are concerned about its future.
>>
>> I propose the addition of a new MREMAP_MIRROR flag to explicitly request
>> this functionality.  The flag simply provides the same functionality as
>> the existing undocumented 'old_size == 0' interface.  As an alternative,
>> we could simply document the 'old_size == 0' interface in the man page.
>> In either case, man page modifications would be needed.
> 
> Right. Adding MREMAP_MIRROR sounds cleaner from application programming
> point of view. But it extends the interface.

Yes.  That is the reason for the RFC.  We currently have functionality
that is not clearly part of a programming interface.  Application programmers
do not like to depend on something that is not part of an interface.

>>
>> Future Direction
>>
>> After more formally adding this to the API (either new flag or documenting
>> existing interface), the mremap code could be enhanced to optimize this
>> case.  Currently, 'mirroring' only sets up the new mapping.  It does not
>> create page table entries for new mapping.  This could be added as an
>> enhancement.
> 
> Then how it achieves mirroring, both the pointers should see the same
> data, that can happen with page table entries pointing to same pages,
> right ?

Correct.

In the code today, page tables for the new (mirrored) mapping are created
as needed via faults.  The enhancement would be to create page table entries
for the new mapping.

-- 
Mike Kravetz

[toc] | [prev] | [next] | [standalone]

#1683127

On 07/06/2017 09:47 PM, Mike Kravetz wrote:
> The mremap system call has the ability to 'mirror' parts of an existing
> mapping.  To do so, it creates a new mapping that maps the same pages as
> the original mapping, just at a different virtual address.  This
> functionality has existed since at least the 2.6 kernel [1].  A comment
> was added to the code to help preserve this feature.

In mremap() implementation move_vma() attempts to do do_unmap() after
move_page_tables(). do_unmap() on the original VMA bails out because
the requested length being 0. Hence both the original VMA and the new
VMA remains after the page table migration. Seems like this whole
mirror function is by coincidence or it has been designed that way ?

[toc] | [prev] | [next] | [standalone]

#1683328

On 07/07/2017 04:03 AM, Anshuman Khandual wrote:
> On 07/06/2017 09:47 PM, Mike Kravetz wrote:
>> The mremap system call has the ability to 'mirror' parts of an existing
>> mapping.  To do so, it creates a new mapping that maps the same pages as
>> the original mapping, just at a different virtual address.  This
>> functionality has existed since at least the 2.6 kernel [1].  A comment
>> was added to the code to help preserve this feature.
> 
> In mremap() implementation move_vma() attempts to do do_unmap() after
> move_page_tables(). do_unmap() on the original VMA bails out because
> the requested length being 0. Hence both the original VMA and the new
> VMA remains after the page table migration. Seems like this whole
> mirror function is by coincidence or it has been designed that way ?

I honestly do not know.  From what I can tell, the functionality existed
in 2.4.  The email thread [1], exists because it was 'accidentally' removed
in 2.6.  All of this is before git history (and my involvement).

My 'guess' is that this functionality was created by coincidence.  Someone
noticed it and took advantage of it.  When it was removed, their code broke. 
The code was 'fixed' and a comment was added to the code in an attempt to
prevent removing the functionality in the future.  Again, this is speculation
as I was not originally involved.

The point of this RFC is to consider adding the functionality to the API.
If we are carrying the functionality in the code, we should at least document
so that application programmers can take advantage of it.

[1] https://lkml.org/lkml/2004/1/12/260
-- 
Mike Kravetz

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web