Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1675068 > unrolled thread

[RFC PATCH] userfaultfd: Add feature to request for a signal delivery

Back to article view | Back to linux.kernel

Contents

  [RFC PATCH] userfaultfd: Add feature to request for a signal delivery Prakash Sangappa <prakash.sangappa@oracle.com> - 2017-06-26 21:50 +0200
    Re: [RFC PATCH] userfaultfd: Add feature to request for a signal  delivery Michal Hocko <mhocko@kernel.org> - 2017-06-27 09:10 +0200
      Re: [RFC PATCH] userfaultfd: Add feature to request for a signal  delivery Mike Rapoport <rppt@linux.vnet.ibm.com> - 2017-06-27 17:40 +0200
        Re: [RFC PATCH] userfaultfd: Add feature to request for a signal  delivery Prakash Sangappa <prakash.sangappa@oracle.com> - 2017-06-27 18:10 +0200
          Re: [RFC PATCH] userfaultfd: Add feature to request for a signal  delivery Mike Rapoport <rppt@linux.vnet.ibm.com> - 2017-06-28 15:20 +0200
            Re: [RFC PATCH] userfaultfd: Add feature to request for a signal  delivery Prakash Sangappa <prakash.sangappa@oracle.com> - 2017-06-28 20:30 +0200
              Re: [RFC PATCH] userfaultfd: Add feature to request for a signal  delivery Michal Hocko <mhocko@kernel.org> - 2017-06-29 10:10 +0200
                Re: [RFC PATCH] userfaultfd: Add feature to request for a signal  delivery "prakash.sangappa" <prakash.sangappa@oracle.com> - 2017-06-29 23:50 +0200
                  Re: [RFC PATCH] userfaultfd: Add feature to request for a signal  delivery Michal Hocko <mhocko@kernel.org> - 2017-06-30 11:50 +0200
                    Re: [RFC PATCH] userfaultfd: Add feature to request for a signal  delivery Andrea Arcangeli <aarcange@redhat.com> - 2017-06-30 15:10 +0200
                      Re: [RFC PATCH] userfaultfd: Add feature to request for a signal  delivery prakash sangappa <prakash.sangappa@oracle.com> - 2017-07-01 03:00 +0200
              Re: [RFC PATCH] userfaultfd: Add feature to request for a signal  delivery Mike Rapoport <rppt@linux.vnet.ibm.com> - 2017-06-29 12:50 +0200
                Re: [RFC PATCH] userfaultfd: Add feature to request for a signal  delivery "prakash.sangappa" <prakash.sangappa@oracle.com> - 2017-06-29 23:50 +0200
      Re: [RFC PATCH] userfaultfd: Add feature to request for a signal  delivery Prakash Sangappa <prakash.sangappa@oracle.com> - 2017-06-27 17:50 +0200

#1675068 — [RFC PATCH] userfaultfd: Add feature to request for a signal delivery

In some cases, userfaultfd mechanism should just deliver a SIGBUS signal
to the faulting process, instead of the page-fault event. Dealing with
page-fault event using a monitor thread can be an overhead in these
cases. For example applications like the database could use the signaling
mechanism for robustness purpose.

Database uses hugetlbfs for performance reason. Files on hugetlbfs
filesystem are created and huge pages allocated using fallocate() API.
Pages are deallocated/freed using fallocate() hole punching support.
These files are mmapped and accessed by many processes as shared memory.
The database keeps track of which offsets in the hugetlbfs file have
pages allocated.

Any access to mapped address over holes in the file, which can occur due
to bugs in the application, is considered invalid and expect the process
to simply receive a SIGBUS.  However, currently when a hole in the file is
accessed via the mapped address, kernel/mm attempts to automatically
allocate a page at page fault time, resulting in implicitly filling the
hole in the file. This may not be the desired behavior for applications
like the database that want to explicitly manage page allocations of
hugetlbfs files.

Using userfaultfd mechanism, with this support to get a signal, database
application can prevent pages from being allocated implicitly when
processes access mapped address over holes in the file.

This patch adds the feature to request for a SIGBUS signal to userfaultfd
mechanism.

See following for previous discussion about the database requirement
leading to this proposal as suggested by Andrea.

http://www.spinics.net/lists/linux-mm/msg129224.html

Signed-off-by: Prakash <prakash.sangappa@oracle.com>
---
  fs/userfaultfd.c                 |  5 +++++
  include/uapi/linux/userfaultfd.h | 10 +++++++++-
  2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c
index 1d622f2..5686d6d2 100644
--- a/fs/userfaultfd.c
+++ b/fs/userfaultfd.c
@@ -371,6 +371,11 @@ int handle_userfault(struct vm_fault *vmf, unsigned 
long reason)
      VM_BUG_ON(reason & ~(VM_UFFD_MISSING|VM_UFFD_WP));
      VM_BUG_ON(!(reason & VM_UFFD_MISSING) ^ !!(reason & VM_UFFD_WP));

+    if (ctx->features & UFFD_FEATURE_SIGBUS) {
+        goto out;
+    }
+
      /*
       * If it's already released don't get it. This avoids to loop
       * in __get_user_pages if userfaultfd_release waits on the
diff --git a/include/uapi/linux/userfaultfd.h 
b/include/uapi/linux/userfaultfd.h
index 3b05953..d39d5db 100644
--- a/include/uapi/linux/userfaultfd.h
+++ b/include/uapi/linux/userfaultfd.h
@@ -23,7 +23,8 @@
                 UFFD_FEATURE_EVENT_REMOVE |    \
                 UFFD_FEATURE_EVENT_UNMAP |        \
                 UFFD_FEATURE_MISSING_HUGETLBFS |    \
-               UFFD_FEATURE_MISSING_SHMEM)
+               UFFD_FEATURE_MISSING_SHMEM |        \
+               UFFD_FEATURE_SIGBUS)
  #define UFFD_API_IOCTLS                \
      ((__u64)1 << _UFFDIO_REGISTER |        \
       (__u64)1 << _UFFDIO_UNREGISTER |    \
@@ -153,6 +154,12 @@ struct uffdio_api {
       * UFFD_FEATURE_MISSING_SHMEM works the same as
       * UFFD_FEATURE_MISSING_HUGETLBFS, but it applies to shmem
       * (i.e. tmpfs and other shmem based APIs).
+     *
+     * UFFD_FEATURE_SIGBUS feature means no page-fault
+     * (UFFD_EVENT_PAGEFAULT) event will be delivered, instead
+     * a SIGBUS signal will be sent to the faulting process.
+     * The application process can enable this behavior by adding
+     * it to uffdio_api.features.
       */
  #define UFFD_FEATURE_PAGEFAULT_FLAG_WP        (1<<0)
  #define UFFD_FEATURE_EVENT_FORK            (1<<1)
@@ -161,6 +168,7 @@ struct uffdio_api {
  #define UFFD_FEATURE_MISSING_HUGETLBFS        (1<<4)
  #define UFFD_FEATURE_MISSING_SHMEM        (1<<5)
  #define UFFD_FEATURE_EVENT_UNMAP        (1<<6)
+#define UFFD_FEATURE_SIGBUS            (1<<7)
      __u64 features;

      __u64 ioctls;
-- 
2.7.4

[toc] | [next] | [standalone]

#1675345 — Re: [RFC PATCH] userfaultfd: Add feature to request for a signal delivery

This is an user visible API so let's CC linux-api mailing list.

On Mon 26-06-17 12:46:13, Prakash Sangappa wrote:
> In some cases, userfaultfd mechanism should just deliver a SIGBUS signal
> to the faulting process, instead of the page-fault event. Dealing with
> page-fault event using a monitor thread can be an overhead in these
> cases. For example applications like the database could use the signaling
> mechanism for robustness purpose.

this is rather confusing. What is the reason that the monitor would be
slower than signal delivery and handling?

> Database uses hugetlbfs for performance reason. Files on hugetlbfs
> filesystem are created and huge pages allocated using fallocate() API.
> Pages are deallocated/freed using fallocate() hole punching support.
> These files are mmapped and accessed by many processes as shared memory.
> The database keeps track of which offsets in the hugetlbfs file have
> pages allocated.
> 
> Any access to mapped address over holes in the file, which can occur due
> to bugs in the application, is considered invalid and expect the process
> to simply receive a SIGBUS.  However, currently when a hole in the file is
> accessed via the mapped address, kernel/mm attempts to automatically
> allocate a page at page fault time, resulting in implicitly filling the
> hole in the file. This may not be the desired behavior for applications
> like the database that want to explicitly manage page allocations of
> hugetlbfs files.

So you register UFFD_FEATURE_SIGBUS on each region tha you are unmapping
and than just let those offenders die?

> Using userfaultfd mechanism, with this support to get a signal, database
> application can prevent pages from being allocated implicitly when
> processes access mapped address over holes in the file.
> 
> This patch adds the feature to request for a SIGBUS signal to userfaultfd
> mechanism.
> 
> See following for previous discussion about the database requirement
> leading to this proposal as suggested by Andrea.
> 
> http://www.spinics.net/lists/linux-mm/msg129224.html

Please make those requirements part of the changelog.

> Signed-off-by: Prakash <prakash.sangappa@oracle.com>
> ---
>  fs/userfaultfd.c                 |  5 +++++
>  include/uapi/linux/userfaultfd.h | 10 +++++++++-
>  2 files changed, 14 insertions(+), 1 deletion(-)
> 
> diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c
> index 1d622f2..5686d6d2 100644
> --- a/fs/userfaultfd.c
> +++ b/fs/userfaultfd.c
> @@ -371,6 +371,11 @@ int handle_userfault(struct vm_fault *vmf, unsigned
> long reason)
>      VM_BUG_ON(reason & ~(VM_UFFD_MISSING|VM_UFFD_WP));
>      VM_BUG_ON(!(reason & VM_UFFD_MISSING) ^ !!(reason & VM_UFFD_WP));
> 
> +    if (ctx->features & UFFD_FEATURE_SIGBUS) {
> +        goto out;
> +    }
> +
>      /*
>       * If it's already released don't get it. This avoids to loop
>       * in __get_user_pages if userfaultfd_release waits on the
> diff --git a/include/uapi/linux/userfaultfd.h
> b/include/uapi/linux/userfaultfd.h
> index 3b05953..d39d5db 100644
> --- a/include/uapi/linux/userfaultfd.h
> +++ b/include/uapi/linux/userfaultfd.h
> @@ -23,7 +23,8 @@
>                 UFFD_FEATURE_EVENT_REMOVE |    \
>                 UFFD_FEATURE_EVENT_UNMAP |        \
>                 UFFD_FEATURE_MISSING_HUGETLBFS |    \
> -               UFFD_FEATURE_MISSING_SHMEM)
> +               UFFD_FEATURE_MISSING_SHMEM |        \
> +               UFFD_FEATURE_SIGBUS)
>  #define UFFD_API_IOCTLS                \
>      ((__u64)1 << _UFFDIO_REGISTER |        \
>       (__u64)1 << _UFFDIO_UNREGISTER |    \
> @@ -153,6 +154,12 @@ struct uffdio_api {
>       * UFFD_FEATURE_MISSING_SHMEM works the same as
>       * UFFD_FEATURE_MISSING_HUGETLBFS, but it applies to shmem
>       * (i.e. tmpfs and other shmem based APIs).
> +     *
> +     * UFFD_FEATURE_SIGBUS feature means no page-fault
> +     * (UFFD_EVENT_PAGEFAULT) event will be delivered, instead
> +     * a SIGBUS signal will be sent to the faulting process.
> +     * The application process can enable this behavior by adding
> +     * it to uffdio_api.features.
>       */
>  #define UFFD_FEATURE_PAGEFAULT_FLAG_WP        (1<<0)
>  #define UFFD_FEATURE_EVENT_FORK            (1<<1)
> @@ -161,6 +168,7 @@ struct uffdio_api {
>  #define UFFD_FEATURE_MISSING_HUGETLBFS        (1<<4)
>  #define UFFD_FEATURE_MISSING_SHMEM        (1<<5)
>  #define UFFD_FEATURE_EVENT_UNMAP        (1<<6)
> +#define UFFD_FEATURE_SIGBUS            (1<<7)
>      __u64 features;
> 
>      __u64 ioctls;
> -- 
> 2.7.4
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

-- 
Michal Hocko
SUSE Labs

[toc] | [prev] | [next] | [standalone]

#1675918 — Re: [RFC PATCH] userfaultfd: Add feature to request for a signal delivery

On Tue, Jun 27, 2017 at 09:06:43AM +0200, Michal Hocko wrote:
> This is an user visible API so let's CC linux-api mailing list.
> 
> On Mon 26-06-17 12:46:13, Prakash Sangappa wrote:
> > In some cases, userfaultfd mechanism should just deliver a SIGBUS signal
> > to the faulting process, instead of the page-fault event. Dealing with
> > page-fault event using a monitor thread can be an overhead in these
> > cases. For example applications like the database could use the signaling
> > mechanism for robustness purpose.
> 
> this is rather confusing. What is the reason that the monitor would be
> slower than signal delivery and handling?
> 
> > Database uses hugetlbfs for performance reason. Files on hugetlbfs
> > filesystem are created and huge pages allocated using fallocate() API.
> > Pages are deallocated/freed using fallocate() hole punching support.
> > These files are mmapped and accessed by many processes as shared memory.
> > The database keeps track of which offsets in the hugetlbfs file have
> > pages allocated.
> > 
> > Any access to mapped address over holes in the file, which can occur due
> > to bugs in the application, is considered invalid and expect the process
> > to simply receive a SIGBUS.  However, currently when a hole in the file is
> > accessed via the mapped address, kernel/mm attempts to automatically
> > allocate a page at page fault time, resulting in implicitly filling the
> > hole in the file. This may not be the desired behavior for applications
> > like the database that want to explicitly manage page allocations of
> > hugetlbfs files.
> 
> So you register UFFD_FEATURE_SIGBUS on each region tha you are unmapping
> and than just let those offenders die?
 
If I understand correctly, the database will create the mapping, then it'll
open userfaultfd and register those mappings with the userfault.
Afterwards, when the application accesses a hole userfault will cause
SIGBUS and the application will process it in whatever way it likes, e.g.
just die.

What I don't understand is why won't you use userfault monitor process that
will take care of the page fault events?
It shouldn't be much overhead running it and it can keep track on all the
userfault file descriptors for you and it will allow more versatile error
handling that SIGBUS.

> > Using userfaultfd mechanism, with this support to get a signal, database
> > application can prevent pages from being allocated implicitly when
> > processes access mapped address over holes in the file.
> > 
> > This patch adds the feature to request for a SIGBUS signal to userfaultfd
> > mechanism.
> > 
> > See following for previous discussion about the database requirement
> > leading to this proposal as suggested by Andrea.
> > 
> > http://www.spinics.net/lists/linux-mm/msg129224.html
> 
> Please make those requirements part of the changelog.
> 
> > Signed-off-by: Prakash <prakash.sangappa@oracle.com>
> > ---
> >  fs/userfaultfd.c                 |  5 +++++
> >  include/uapi/linux/userfaultfd.h | 10 +++++++++-
> >  2 files changed, 14 insertions(+), 1 deletion(-)
> > 
> > diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c
> > index 1d622f2..5686d6d2 100644
> > --- a/fs/userfaultfd.c
> > +++ b/fs/userfaultfd.c
> > @@ -371,6 +371,11 @@ int handle_userfault(struct vm_fault *vmf, unsigned
> > long reason)
> >      VM_BUG_ON(reason & ~(VM_UFFD_MISSING|VM_UFFD_WP));
> >      VM_BUG_ON(!(reason & VM_UFFD_MISSING) ^ !!(reason & VM_UFFD_WP));
> > 
> > +    if (ctx->features & UFFD_FEATURE_SIGBUS) {
> > +        goto out;
> > +    }
> > +
> >      /*
> >       * If it's already released don't get it. This avoids to loop
> >       * in __get_user_pages if userfaultfd_release waits on the
> > diff --git a/include/uapi/linux/userfaultfd.h
> > b/include/uapi/linux/userfaultfd.h
> > index 3b05953..d39d5db 100644
> > --- a/include/uapi/linux/userfaultfd.h
> > +++ b/include/uapi/linux/userfaultfd.h
> > @@ -23,7 +23,8 @@
> >                 UFFD_FEATURE_EVENT_REMOVE |    \
> >                 UFFD_FEATURE_EVENT_UNMAP |        \
> >                 UFFD_FEATURE_MISSING_HUGETLBFS |    \
> > -               UFFD_FEATURE_MISSING_SHMEM)
> > +               UFFD_FEATURE_MISSING_SHMEM |        \
> > +               UFFD_FEATURE_SIGBUS)
> >  #define UFFD_API_IOCTLS                \
> >      ((__u64)1 << _UFFDIO_REGISTER |        \
> >       (__u64)1 << _UFFDIO_UNREGISTER |    \
> > @@ -153,6 +154,12 @@ struct uffdio_api {
> >       * UFFD_FEATURE_MISSING_SHMEM works the same as
> >       * UFFD_FEATURE_MISSING_HUGETLBFS, but it applies to shmem
> >       * (i.e. tmpfs and other shmem based APIs).
> > +     *
> > +     * UFFD_FEATURE_SIGBUS feature means no page-fault
> > +     * (UFFD_EVENT_PAGEFAULT) event will be delivered, instead
> > +     * a SIGBUS signal will be sent to the faulting process.
> > +     * The application process can enable this behavior by adding
> > +     * it to uffdio_api.features.
> >       */
> >  #define UFFD_FEATURE_PAGEFAULT_FLAG_WP        (1<<0)
> >  #define UFFD_FEATURE_EVENT_FORK            (1<<1)
> > @@ -161,6 +168,7 @@ struct uffdio_api {
> >  #define UFFD_FEATURE_MISSING_HUGETLBFS        (1<<4)
> >  #define UFFD_FEATURE_MISSING_SHMEM        (1<<5)
> >  #define UFFD_FEATURE_EVENT_UNMAP        (1<<6)
> > +#define UFFD_FEATURE_SIGBUS            (1<<7)
> >      __u64 features;
> > 
> >      __u64 ioctls;
> > -- 
> > 2.7.4
> > 
> -- 
> Michal Hocko
> SUSE Labs
> 
--
Sincerely yours,
Mike.

[toc] | [prev] | [next] | [standalone]

#1675949 — Re: [RFC PATCH] userfaultfd: Add feature to request for a signal delivery

On 6/27/17 8:35 AM, Mike Rapoport wrote:

> On Tue, Jun 27, 2017 at 09:06:43AM +0200, Michal Hocko wrote:
>> This is an user visible API so let's CC linux-api mailing list.
>>
>> On Mon 26-06-17 12:46:13, Prakash Sangappa wrote:
>>> In some cases, userfaultfd mechanism should just deliver a SIGBUS signal
>>> to the faulting process, instead of the page-fault event. Dealing with
>>> page-fault event using a monitor thread can be an overhead in these
>>> cases. For example applications like the database could use the signaling
>>> mechanism for robustness purpose.
>> this is rather confusing. What is the reason that the monitor would be
>> slower than signal delivery and handling?
>>
>>> Database uses hugetlbfs for performance reason. Files on hugetlbfs
>>> filesystem are created and huge pages allocated using fallocate() API.
>>> Pages are deallocated/freed using fallocate() hole punching support.
>>> These files are mmapped and accessed by many processes as shared memory.
>>> The database keeps track of which offsets in the hugetlbfs file have
>>> pages allocated.
>>>
>>> Any access to mapped address over holes in the file, which can occur due
>>> to bugs in the application, is considered invalid and expect the process
>>> to simply receive a SIGBUS.  However, currently when a hole in the file is
>>> accessed via the mapped address, kernel/mm attempts to automatically
>>> allocate a page at page fault time, resulting in implicitly filling the
>>> hole in the file. This may not be the desired behavior for applications
>>> like the database that want to explicitly manage page allocations of
>>> hugetlbfs files.
>> So you register UFFD_FEATURE_SIGBUS on each region tha you are unmapping
>> and than just let those offenders die?
>   
> If I understand correctly, the database will create the mapping, then it'll
> open userfaultfd and register those mappings with the userfault.
> Afterwards, when the application accesses a hole userfault will cause
> SIGBUS and the application will process it in whatever way it likes, e.g.
> just die.

Yes.

> What I don't understand is why won't you use userfault monitor process that
> will take care of the page fault events?
> It shouldn't be much overhead running it and it can keep track on all the
> userfault file descriptors for you and it will allow more versatile error
> handling that SIGBUS.
>

Co-ordination with the external monitor process by all the database 
processes
to send  their userfaultfd is still an overhead.


>>> Using userfaultfd mechanism, with this support to get a signal, database
>>> application can prevent pages from being allocated implicitly when
>>> processes access mapped address over holes in the file.
>>>
>>> This patch adds the feature to request for a SIGBUS signal to userfaultfd
>>> mechanism.
>>>
>>> See following for previous discussion about the database requirement
>>> leading to this proposal as suggested by Andrea.
>>>
>>> http://www.spinics.net/lists/linux-mm/msg129224.html
>> Please make those requirements part of the changelog.
>>
>>> Signed-off-by: Prakash <prakash.sangappa@oracle.com>
>>> ---
>>>   fs/userfaultfd.c                 |  5 +++++
>>>   include/uapi/linux/userfaultfd.h | 10 +++++++++-
>>>   2 files changed, 14 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c
>>> index 1d622f2..5686d6d2 100644
>>> --- a/fs/userfaultfd.c
>>> +++ b/fs/userfaultfd.c
>>> @@ -371,6 +371,11 @@ int handle_userfault(struct vm_fault *vmf, unsigned
>>> long reason)
>>>       VM_BUG_ON(reason & ~(VM_UFFD_MISSING|VM_UFFD_WP));
>>>       VM_BUG_ON(!(reason & VM_UFFD_MISSING) ^ !!(reason & VM_UFFD_WP));
>>>
>>> +    if (ctx->features & UFFD_FEATURE_SIGBUS) {
>>> +        goto out;
>>> +    }
>>> +
>>>       /*
>>>        * If it's already released don't get it. This avoids to loop
>>>        * in __get_user_pages if userfaultfd_release waits on the
>>> diff --git a/include/uapi/linux/userfaultfd.h
>>> b/include/uapi/linux/userfaultfd.h
>>> index 3b05953..d39d5db 100644
>>> --- a/include/uapi/linux/userfaultfd.h
>>> +++ b/include/uapi/linux/userfaultfd.h
>>> @@ -23,7 +23,8 @@
>>>                  UFFD_FEATURE_EVENT_REMOVE |    \
>>>                  UFFD_FEATURE_EVENT_UNMAP |        \
>>>                  UFFD_FEATURE_MISSING_HUGETLBFS |    \
>>> -               UFFD_FEATURE_MISSING_SHMEM)
>>> +               UFFD_FEATURE_MISSING_SHMEM |        \
>>> +               UFFD_FEATURE_SIGBUS)
>>>   #define UFFD_API_IOCTLS                \
>>>       ((__u64)1 << _UFFDIO_REGISTER |        \
>>>        (__u64)1 << _UFFDIO_UNREGISTER |    \
>>> @@ -153,6 +154,12 @@ struct uffdio_api {
>>>        * UFFD_FEATURE_MISSING_SHMEM works the same as
>>>        * UFFD_FEATURE_MISSING_HUGETLBFS, but it applies to shmem
>>>        * (i.e. tmpfs and other shmem based APIs).
>>> +     *
>>> +     * UFFD_FEATURE_SIGBUS feature means no page-fault
>>> +     * (UFFD_EVENT_PAGEFAULT) event will be delivered, instead
>>> +     * a SIGBUS signal will be sent to the faulting process.
>>> +     * The application process can enable this behavior by adding
>>> +     * it to uffdio_api.features.
>>>        */
>>>   #define UFFD_FEATURE_PAGEFAULT_FLAG_WP        (1<<0)
>>>   #define UFFD_FEATURE_EVENT_FORK            (1<<1)
>>> @@ -161,6 +168,7 @@ struct uffdio_api {
>>>   #define UFFD_FEATURE_MISSING_HUGETLBFS        (1<<4)
>>>   #define UFFD_FEATURE_MISSING_SHMEM        (1<<5)
>>>   #define UFFD_FEATURE_EVENT_UNMAP        (1<<6)
>>> +#define UFFD_FEATURE_SIGBUS            (1<<7)
>>>       __u64 features;
>>>
>>>       __u64 ioctls;
>>> -- 
>>> 2.7.4
>>>
>> -- 
>> Michal Hocko
>> SUSE Labs
>>
> --
> Sincerely yours,
> Mike.
>

[toc] | [prev] | [next] | [standalone]

#1676649 — Re: [RFC PATCH] userfaultfd: Add feature to request for a signal delivery

On Tue, Jun 27, 2017 at 09:01:20AM -0700, Prakash Sangappa wrote:
> On 6/27/17 8:35 AM, Mike Rapoport wrote:
> 
> >On Tue, Jun 27, 2017 at 09:06:43AM +0200, Michal Hocko wrote:
> >>This is an user visible API so let's CC linux-api mailing list.
> >>
> >>On Mon 26-06-17 12:46:13, Prakash Sangappa wrote:
> >>>In some cases, userfaultfd mechanism should just deliver a SIGBUS signal
> >>>to the faulting process, instead of the page-fault event. Dealing with
> >>>page-fault event using a monitor thread can be an overhead in these
> >>>cases. For example applications like the database could use the signaling
> >>>mechanism for robustness purpose.
> >>this is rather confusing. What is the reason that the monitor would be
> >>slower than signal delivery and handling?
> >>
> >>>Database uses hugetlbfs for performance reason. Files on hugetlbfs
> >>>filesystem are created and huge pages allocated using fallocate() API.
> >>>Pages are deallocated/freed using fallocate() hole punching support.
> >>>These files are mmapped and accessed by many processes as shared memory.
> >>>The database keeps track of which offsets in the hugetlbfs file have
> >>>pages allocated.
> >>>
> >>>Any access to mapped address over holes in the file, which can occur due
> >>>to bugs in the application, is considered invalid and expect the process
> >>>to simply receive a SIGBUS.  However, currently when a hole in the file is
> >>>accessed via the mapped address, kernel/mm attempts to automatically
> >>>allocate a page at page fault time, resulting in implicitly filling the
> >>>hole in the file. This may not be the desired behavior for applications
> >>>like the database that want to explicitly manage page allocations of
> >>>hugetlbfs files.
> >>So you register UFFD_FEATURE_SIGBUS on each region tha you are unmapping
> >>and than just let those offenders die?
> >If I understand correctly, the database will create the mapping, then it'll
> >open userfaultfd and register those mappings with the userfault.
> >Afterwards, when the application accesses a hole userfault will cause
> >SIGBUS and the application will process it in whatever way it likes, e.g.
> >just die.
> 
> Yes.
>
> >What I don't understand is why won't you use userfault monitor process that
> >will take care of the page fault events?
> >It shouldn't be much overhead running it and it can keep track on all the
> >userfault file descriptors for you and it will allow more versatile error
> >handling that SIGBUS.
> >
> 
> Co-ordination with the external monitor process by all the database
> processes
> to send  their userfaultfd is still an overhead.

You are planning to register in userfaultfd only the holes you punch to
deallocate pages, am I right?

And the co-ordination of the userfault file descriptor with the monitor
would have been added after calls to fallocate() and userfaultfd_register()?

I've just been thinking that maybe it would be possible to use
UFFD_EVENT_REMOVE for this case. We anyway need to implement the generation
of UFFD_EVENT_REMOVE for the case of hole punching in hugetlbfs for
non-cooperative userfaultfd. It could be that it will solve your issue as
well.

> >>>Using userfaultfd mechanism, with this support to get a signal, database
> >>>application can prevent pages from being allocated implicitly when
> >>>processes access mapped address over holes in the file.
> >>>
> >>>This patch adds the feature to request for a SIGBUS signal to userfaultfd
> >>>mechanism.
> >>>
> >>>See following for previous discussion about the database requirement
> >>>leading to this proposal as suggested by Andrea.
> >>>
> >>>http://www.spinics.net/lists/linux-mm/msg129224.html
> >>Please make those requirements part of the changelog.
> >>
> >>>Signed-off-by: Prakash <prakash.sangappa@oracle.com>
> >>>---
> >>>  fs/userfaultfd.c                 |  5 +++++
> >>>  include/uapi/linux/userfaultfd.h | 10 +++++++++-
> >>>  2 files changed, 14 insertions(+), 1 deletion(-)
> >>>
> >>>diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c
> >>>index 1d622f2..5686d6d2 100644
> >>>--- a/fs/userfaultfd.c
> >>>+++ b/fs/userfaultfd.c
> >>>@@ -371,6 +371,11 @@ int handle_userfault(struct vm_fault *vmf, unsigned
> >>>long reason)
> >>>      VM_BUG_ON(reason & ~(VM_UFFD_MISSING|VM_UFFD_WP));
> >>>      VM_BUG_ON(!(reason & VM_UFFD_MISSING) ^ !!(reason & VM_UFFD_WP));
> >>>
> >>>+    if (ctx->features & UFFD_FEATURE_SIGBUS) {
> >>>+        goto out;
> >>>+    }
> >>>+
> >>>      /*
> >>>       * If it's already released don't get it. This avoids to loop
> >>>       * in __get_user_pages if userfaultfd_release waits on the
> >>>diff --git a/include/uapi/linux/userfaultfd.h
> >>>b/include/uapi/linux/userfaultfd.h
> >>>index 3b05953..d39d5db 100644
> >>>--- a/include/uapi/linux/userfaultfd.h
> >>>+++ b/include/uapi/linux/userfaultfd.h
> >>>@@ -23,7 +23,8 @@
> >>>                 UFFD_FEATURE_EVENT_REMOVE |    \
> >>>                 UFFD_FEATURE_EVENT_UNMAP |        \
> >>>                 UFFD_FEATURE_MISSING_HUGETLBFS |    \
> >>>-               UFFD_FEATURE_MISSING_SHMEM)
> >>>+               UFFD_FEATURE_MISSING_SHMEM |        \
> >>>+               UFFD_FEATURE_SIGBUS)
> >>>  #define UFFD_API_IOCTLS                \
> >>>      ((__u64)1 << _UFFDIO_REGISTER |        \
> >>>       (__u64)1 << _UFFDIO_UNREGISTER |    \
> >>>@@ -153,6 +154,12 @@ struct uffdio_api {
> >>>       * UFFD_FEATURE_MISSING_SHMEM works the same as
> >>>       * UFFD_FEATURE_MISSING_HUGETLBFS, but it applies to shmem
> >>>       * (i.e. tmpfs and other shmem based APIs).
> >>>+     *
> >>>+     * UFFD_FEATURE_SIGBUS feature means no page-fault
> >>>+     * (UFFD_EVENT_PAGEFAULT) event will be delivered, instead
> >>>+     * a SIGBUS signal will be sent to the faulting process.
> >>>+     * The application process can enable this behavior by adding
> >>>+     * it to uffdio_api.features.
> >>>       */
> >>>  #define UFFD_FEATURE_PAGEFAULT_FLAG_WP        (1<<0)
> >>>  #define UFFD_FEATURE_EVENT_FORK            (1<<1)
> >>>@@ -161,6 +168,7 @@ struct uffdio_api {
> >>>  #define UFFD_FEATURE_MISSING_HUGETLBFS        (1<<4)
> >>>  #define UFFD_FEATURE_MISSING_SHMEM        (1<<5)
> >>>  #define UFFD_FEATURE_EVENT_UNMAP        (1<<6)
> >>>+#define UFFD_FEATURE_SIGBUS            (1<<7)
> >>>      __u64 features;
> >>>
> >>>      __u64 ioctls;
> >>>-- 
> >>>2.7.4
> >>>
> >>-- 
> >>Michal Hocko
> >>SUSE Labs
> >>
> >--
> >Sincerely yours,
> >Mike.
> >
> 

[toc] | [prev] | [next] | [standalone]

#1677047 — Re: [RFC PATCH] userfaultfd: Add feature to request for a signal delivery

On 6/28/17 6:18 AM, Mike Rapoport wrote:
> On Tue, Jun 27, 2017 at 09:01:20AM -0700, Prakash Sangappa wrote:
>> On 6/27/17 8:35 AM, Mike Rapoport wrote:
>>
>>> On Tue, Jun 27, 2017 at 09:06:43AM +0200, Michal Hocko wrote:
>>>> This is an user visible API so let's CC linux-api mailing list.
>>>>
>>>> On Mon 26-06-17 12:46:13, Prakash Sangappa wrote:
>>>>
>>>>> Any access to mapped address over holes in the file, which can occur due
>>>>> to bugs in the application, is considered invalid and expect the process
>>>>> to simply receive a SIGBUS.  However, currently when a hole in the file is
>>>>> accessed via the mapped address, kernel/mm attempts to automatically
>>>>> allocate a page at page fault time, resulting in implicitly filling the
>>>>> hole in the file. This may not be the desired behavior for applications
>>>>> like the database that want to explicitly manage page allocations of
>>>>> hugetlbfs files.
>>>> So you register UFFD_FEATURE_SIGBUS on each region tha you are unmapping
>>>> and than just let those offenders die?
>>> If I understand correctly, the database will create the mapping, then it'll
>>> open userfaultfd and register those mappings with the userfault.
>>> Afterwards, when the application accesses a hole userfault will cause
>>> SIGBUS and the application will process it in whatever way it likes, e.g.
>>> just die.
>> Yes.
>>
>>> What I don't understand is why won't you use userfault monitor process that
>>> will take care of the page fault events?
>>> It shouldn't be much overhead running it and it can keep track on all the
>>> userfault file descriptors for you and it will allow more versatile error
>>> handling that SIGBUS.
>>>
>> Co-ordination with the external monitor process by all the database
>> processes
>> to send  their userfaultfd is still an overhead.
> You are planning to register in userfaultfd only the holes you punch to
> deallocate pages, am I right?


No, the entire mmap'ed region. The DB processes would mmap(MAP_NORESERVE)
hugetlbfs files, register this mapped address with userfaultfd ones 
right after
the mmap() call.

>
> And the co-ordination of the userfault file descriptor with the monitor
> would have been added after calls to fallocate() and userfaultfd_register()?

Well, the database application does not need to deal with a monitor.

>
> I've just been thinking that maybe it would be possible to use
> UFFD_EVENT_REMOVE for this case. We anyway need to implement the generation
> of UFFD_EVENT_REMOVE for the case of hole punching in hugetlbfs for
> non-cooperative userfaultfd. It could be that it will solve your issue as
> well.
>

Will this result in a signal delivery?

In the use case described, the database application does not need any event
for  hole punching. Basically, just a signal for any invalid access to 
mapped
area over holes in the file.

[toc] | [prev] | [next] | [standalone]

#1677508 — Re: [RFC PATCH] userfaultfd: Add feature to request for a signal delivery

On Wed 28-06-17 11:23:32, Prakash Sangappa wrote:
> 
> 
> On 6/28/17 6:18 AM, Mike Rapoport wrote:
[...]
> >I've just been thinking that maybe it would be possible to use
> >UFFD_EVENT_REMOVE for this case. We anyway need to implement the generation
> >of UFFD_EVENT_REMOVE for the case of hole punching in hugetlbfs for
> >non-cooperative userfaultfd. It could be that it will solve your issue as
> >well.
> >
> 
> Will this result in a signal delivery?
> 
> In the use case described, the database application does not need any event
> for  hole punching. Basically, just a signal for any invalid access to
> mapped area over holes in the file.

OK, but it would be better to think that through for other potential
usecases so that this doesn't end up as a single hugetlb feature. E.g.
what should happen if a regular anonymous memory gets swapped out?
Should we deliver signal as well? How does userspace tell whether this
was a no backing page from unavailable backing page?
-- 
Michal Hocko
SUSE Labs

[toc] | [prev] | [next] | [standalone]

#1678230 — Re: [RFC PATCH] userfaultfd: Add feature to request for a signal delivery

On 06/29/2017 01:09 AM, Michal Hocko wrote:
> On Wed 28-06-17 11:23:32, Prakash Sangappa wrote:
>>
>> On 6/28/17 6:18 AM, Mike Rapoport wrote:
> [...]
>>> I've just been thinking that maybe it would be possible to use
>>> UFFD_EVENT_REMOVE for this case. We anyway need to implement the generation
>>> of UFFD_EVENT_REMOVE for the case of hole punching in hugetlbfs for
>>> non-cooperative userfaultfd. It could be that it will solve your issue as
>>> well.
>>>
>> Will this result in a signal delivery?
>>
>> In the use case described, the database application does not need any event
>> for  hole punching. Basically, just a signal for any invalid access to
>> mapped area over holes in the file.
> OK, but it would be better to think that through for other potential
> usecases so that this doesn't end up as a single hugetlb feature. E.g.
> what should happen if a regular anonymous memory gets swapped out?
> Should we deliver signal as well? How does userspace tell whether this
> was a no backing page from unavailable backing page?

This may not be useful in all cases. Potential, it could be used
with use of mlock() on anonymous memory to ensure any access
to memory that is not locked is caught, again for robustness
purpose.

[toc] | [prev] | [next] | [standalone]

#1678637 — Re: [RFC PATCH] userfaultfd: Add feature to request for a signal delivery

[CC John, the thread started
http://lkml.kernel.org/r/9363561f-a9cd-7ab6-9c11-ab9a99dc89f1@oracle.com]

On Thu 29-06-17 14:41:22, prakash.sangappa wrote:
> 
> 
> On 06/29/2017 01:09 AM, Michal Hocko wrote:
> >On Wed 28-06-17 11:23:32, Prakash Sangappa wrote:
> >>
> >>On 6/28/17 6:18 AM, Mike Rapoport wrote:
> >[...]
> >>>I've just been thinking that maybe it would be possible to use
> >>>UFFD_EVENT_REMOVE for this case. We anyway need to implement the generation
> >>>of UFFD_EVENT_REMOVE for the case of hole punching in hugetlbfs for
> >>>non-cooperative userfaultfd. It could be that it will solve your issue as
> >>>well.
> >>>
> >>Will this result in a signal delivery?
> >>
> >>In the use case described, the database application does not need any event
> >>for  hole punching. Basically, just a signal for any invalid access to
> >>mapped area over holes in the file.
> >OK, but it would be better to think that through for other potential
> >usecases so that this doesn't end up as a single hugetlb feature. E.g.
> >what should happen if a regular anonymous memory gets swapped out?
> >Should we deliver signal as well? How does userspace tell whether this
> >was a no backing page from unavailable backing page?
> 
> This may not be useful in all cases. Potential, it could be used
> with use of mlock() on anonymous memory to ensure any access
> to memory that is not locked is caught, again for robustness
> purpose.

The thing I wanted to point out is that not only this should be a single
usecase thing (I believe others will pop out as well - see below) but it
should also be well defined as this is a user visible API. Please try to
write a patch to the userfaultfd man page to clarify the exact semantic.
This should help the further discussion.

As an aside, I rememeber that prior to MADV_FREE there was long
discussion about lazy freeing of memory from userspace. Some users
wanted to be signalled when their memory was freed by the system so that
they could rebuild the original content (e.g. uncompressed images in
memory). It seems like MADV_FREE + this signalling could be used for
that usecase. John would surely know more about those usecases.
-- 
Michal Hocko
SUSE Labs

[toc] | [prev] | [next] | [standalone]

#1678799 — Re: [RFC PATCH] userfaultfd: Add feature to request for a signal delivery

On Fri, Jun 30, 2017 at 11:47:35AM +0200, Michal Hocko wrote:
> [CC John, the thread started
> http://lkml.kernel.org/r/9363561f-a9cd-7ab6-9c11-ab9a99dc89f1@oracle.com]
> 
> On Thu 29-06-17 14:41:22, prakash.sangappa wrote:
> > 
> > 
> > On 06/29/2017 01:09 AM, Michal Hocko wrote:
> > >On Wed 28-06-17 11:23:32, Prakash Sangappa wrote:
> > >>
> > >>On 6/28/17 6:18 AM, Mike Rapoport wrote:
> > >[...]
> > >>>I've just been thinking that maybe it would be possible to use
> > >>>UFFD_EVENT_REMOVE for this case. We anyway need to implement the generation
> > >>>of UFFD_EVENT_REMOVE for the case of hole punching in hugetlbfs for
> > >>>non-cooperative userfaultfd. It could be that it will solve your issue as
> > >>>well.
> > >>>
> > >>Will this result in a signal delivery?
> > >>
> > >>In the use case described, the database application does not need any event
> > >>for  hole punching. Basically, just a signal for any invalid access to
> > >>mapped area over holes in the file.
> > >OK, but it would be better to think that through for other potential
> > >usecases so that this doesn't end up as a single hugetlb feature. E.g.
> > >what should happen if a regular anonymous memory gets swapped out?
> > >Should we deliver signal as well? How does userspace tell whether this
> > >was a no backing page from unavailable backing page?
> > 
> > This may not be useful in all cases. Potential, it could be used
> > with use of mlock() on anonymous memory to ensure any access
> > to memory that is not locked is caught, again for robustness
> > purpose.
> 
> The thing I wanted to point out is that not only this should be a single
> usecase thing (I believe others will pop out as well - see below) but it
> should also be well defined as this is a user visible API. Please try to
> write a patch to the userfaultfd man page to clarify the exact semantic.
> This should help the further discussion.
> 
> As an aside, I rememeber that prior to MADV_FREE there was long
> discussion about lazy freeing of memory from userspace. Some users
> wanted to be signalled when their memory was freed by the system so that
> they could rebuild the original content (e.g. uncompressed images in
> memory). It seems like MADV_FREE + this signalling could be used for
> that usecase. John would surely know more about those usecases.

That would provide an equivalent API to the one volatile pages
provided agreed. So it would allow to adapt code (if any?) more easily
to drop the duplicate feature in volatile pages code (however it would
be faster if the userland code using volatile pages lazy reclaim mode
was converted to poll the uffd so the kernel talks directly to the
monitor without involving a SIGBUS signal handler which will cause
spurious enter/exit if compared to signal-less uffd API).

The main benefit in my view is not volatile pages but that
UFFD_FEATURE_SIGBUS would work equally well to enforce robustness on
all kind of memory not only hugetlbfs (so one could run the database
with robustness on THP over tmpfs) and the new cache can be injected
in the filesystem using UFFDIO_COPY which is likely faster than
fallocate as UFFDIO_COPY was already demonstrated to be faster even
than a regular page fault.

It's also simpler to handle backwards compatibility with the
UFFDIO_API call, that allows probing if UFFD_FEATURE_SIGBUS is
supported by the running kernel regardless of kernel version (so it
can be backported and enabled by the database, without the database
noticing it's on a older kernel version).

So while this wasn't the intended way to use the userfault and I
already pointed out the possibility to use a single monitor to do all
this, I'm positive about UFFD_FEATURE_SIGBUS if the overhead of having
a monitor is so concerning.

Ultimately there are many pros and just a single cons: the branch in
handle_userfault().

I wonder if it would be possible to use static_branch_enable() in
UFFDIO_API and static_branch_unlikely in handle_userfault() to
eliminate that branch but perhaps it's overkill and UFFDIO_API is
unprivileged and it would send an IPI to all CPUs. I don't think we
normally expose the static_branch_enable() to unprivileged userland
and making UFFD_FEATURE_SIGBUS a privileged op doesn't sound
attractive (although the alternative of altering a hugetlbfs mount
option would be a privileged op).

Thanks,
Andrea

[toc] | [prev] | [next] | [standalone]

#1679227 — Re: [RFC PATCH] userfaultfd: Add feature to request for a signal delivery

On 6/30/2017 6:08 AM, Andrea Arcangeli wrote:
> On Fri, Jun 30, 2017 at 11:47:35AM +0200, Michal Hocko wrote:
[...]
>> As an aside, I rememeber that prior to MADV_FREE there was long
>> discussion about lazy freeing of memory from userspace. Some users
>> wanted to be signalled when their memory was freed by the system so that
>> they could rebuild the original content (e.g. uncompressed images in
>> memory). It seems like MADV_FREE + this signalling could be used for
>> that usecase. John would surely know more about those usecases.
> That would provide an equivalent API to the one volatile pages
> provided agreed. So it would allow to adapt code (if any?) more easily
> to drop the duplicate feature in volatile pages code (however it would
> be faster if the userland code using volatile pages lazy reclaim mode
> was converted to poll the uffd so the kernel talks directly to the
> monitor without involving a SIGBUS signal handler which will cause
> spurious enter/exit if compared to signal-less uffd API).
>
> The main benefit in my view is not volatile pages but that
> UFFD_FEATURE_SIGBUS would work equally well to enforce robustness on
> all kind of memory not only hugetlbfs (so one could run the database
> with robustness on THP over tmpfs) and the new cache can be injected
> in the filesystem using UFFDIO_COPY which is likely faster than
> fallocate as UFFDIO_COPY was already demonstrated to be faster even
> than a regular page fault.

Interesting that UFFDIO_COPY is faster then fallocate().  In the DB use case
the page does not need to be allocated at the time a process trips on 
the hugetlbfs
file hole and receives SIGBUS.  fallocate() is called on the hugetlbfs file,
when more memory needs to be allocated by a separate process.

> It's also simpler to handle backwards compatibility with the
> UFFDIO_API call, that allows probing if UFFD_FEATURE_SIGBUS is
> supported by the running kernel regardless of kernel version (so it
> can be backported and enabled by the database, without the database
> noticing it's on a older kernel version).

Yes, this is useful as this change will need to be back ported.

> So while this wasn't the intended way to use the userfault and I
> already pointed out the possibility to use a single monitor to do all
> this, I'm positive about UFFD_FEATURE_SIGBUS if the overhead of having
> a monitor is so concerning.
>
> Ultimately there are many pros and just a single cons: the branch in
> handle_userfault().
>
> I wonder if it would be possible to use static_branch_enable() in
> UFFDIO_API and static_branch_unlikely in handle_userfault() to
> eliminate that branch but perhaps it's overkill and UFFDIO_API is
> unprivileged and it would send an IPI to all CPUs. I don't think we
> normally expose the static_branch_enable() to unprivileged userland
> and making UFFD_FEATURE_SIGBUS a privileged op doesn't sound
> attractive (although the alternative of altering a hugetlbfs mount
> option would be a privileged op).

Regarding hugetlbfs mount option, one consideration is to allow mounts of
hugetlbfs inside user namespaces's mount namespace. Which would allow
non privileged processes to mount hugetlbfs for use inside a user 
namespace.
This may be needed even for the 'min_size' mount option using which an
application could reserve huge pages and mount a filesystem for its use,
with out the need to have privileges given the system has enough hugepages
configured.  It seems if non privileged processes are allowed to mount 
hugetlbfs
filesystem, then min_size should be subject to some resource limits.

Mounting inside user namespace will be a different patch proposal later.


>
> Thanks,
> Andrea

[toc] | [prev] | [next] | [standalone]

#1677597 — Re: [RFC PATCH] userfaultfd: Add feature to request for a signal delivery

On Wed, Jun 28, 2017 at 11:23:32AM -0700, Prakash Sangappa wrote:
> 
> 
> On 6/28/17 6:18 AM, Mike Rapoport wrote:
> >On Tue, Jun 27, 2017 at 09:01:20AM -0700, Prakash Sangappa wrote:
> >>On 6/27/17 8:35 AM, Mike Rapoport wrote:
> >>
> >>>On Tue, Jun 27, 2017 at 09:06:43AM +0200, Michal Hocko wrote:
> >>>>This is an user visible API so let's CC linux-api mailing list.
> >>>>
> >>>>On Mon 26-06-17 12:46:13, Prakash Sangappa wrote:
> >>>>
> >>>>>Any access to mapped address over holes in the file, which can occur due
> >>>>>to bugs in the application, is considered invalid and expect the process
> >>>>>to simply receive a SIGBUS.  However, currently when a hole in the file is
> >>>>>accessed via the mapped address, kernel/mm attempts to automatically
> >>>>>allocate a page at page fault time, resulting in implicitly filling the
> >>>>>hole in the file. This may not be the desired behavior for applications
> >>>>>like the database that want to explicitly manage page allocations of
> >>>>>hugetlbfs files.
> >>>>So you register UFFD_FEATURE_SIGBUS on each region tha you are unmapping
> >>>>and than just let those offenders die?
> >>>If I understand correctly, the database will create the mapping, then it'll
> >>>open userfaultfd and register those mappings with the userfault.
> >>>Afterwards, when the application accesses a hole userfault will cause
> >>>SIGBUS and the application will process it in whatever way it likes, e.g.
> >>>just die.
> >>Yes.
> >>
> >>>What I don't understand is why won't you use userfault monitor process that
> >>>will take care of the page fault events?
> >>>It shouldn't be much overhead running it and it can keep track on all the
> >>>userfault file descriptors for you and it will allow more versatile error
> >>>handling that SIGBUS.
> >>>
> >>Co-ordination with the external monitor process by all the database
> >>processes
> >>to send  their userfaultfd is still an overhead.
> >You are planning to register in userfaultfd only the holes you punch to
> >deallocate pages, am I right?
> 
> 
> No, the entire mmap'ed region. The DB processes would mmap(MAP_NORESERVE)
> hugetlbfs files, register this mapped address with userfaultfd ones right
> after
> the mmap() call.
> 
> >
> >And the co-ordination of the userfault file descriptor with the monitor
> >would have been added after calls to fallocate() and userfaultfd_register()?
> 
> Well, the database application does not need to deal with a monitor.
> 
> >
> >I've just been thinking that maybe it would be possible to use
> >UFFD_EVENT_REMOVE for this case. We anyway need to implement the generation
> >of UFFD_EVENT_REMOVE for the case of hole punching in hugetlbfs for
> >non-cooperative userfaultfd. It could be that it will solve your issue as
> >well.
> >
> 
> Will this result in a signal delivery?
> 
> In the use case described, the database application does not need any event
> for  hole punching. Basically, just a signal for any invalid access to
> mapped
> area over holes in the file.
 
Well, what I had in mind was using a single-process uffd monitor that will
track all the userfault file descriptors. With UFFD_EVENT_REMOVE this
process will know what areas are invalid and it will be able to process the
invalid access in any way it likes, e.g. send SIGBUS to the database
application.

If you mmap() and userfaultfd_register() only at the initialization time,
it might be also possible to avoid sending userfault file descriptors to
the monitor process with UFFD_FEATURE_EVENT_FORK.

--
Sincerely yours,
Mike.

[toc] | [prev] | [next] | [standalone]

#1678225 — Re: [RFC PATCH] userfaultfd: Add feature to request for a signal delivery

On 06/29/2017 03:46 AM, Mike Rapoport wrote:
> On Wed, Jun 28, 2017 at 11:23:32AM -0700, Prakash Sangappa wrote:
[...]
>>
>> Will this result in a signal delivery?
>>
>> In the use case described, the database application does not need any event
>> for  hole punching. Basically, just a signal for any invalid access to
>> mapped
>> area over holes in the file.
>   
> Well, what I had in mind was using a single-process uffd monitor that will
> track all the userfault file descriptors. With UFFD_EVENT_REMOVE this
> process will know what areas are invalid and it will be able to process the
> invalid access in any way it likes, e.g. send SIGBUS to the database
> application.


Use of a monitor process is also an overhead for the database.


>
> If you mmap() and userfaultfd_register() only at the initialization time,
> it might be also possible to avoid sending userfault file descriptors to
> the monitor process with UFFD_FEATURE_EVENT_FORK.

The new processes are always exec'd in the database case and these
processes could be mapping different files. So, not sure if
UFFD_FEATURE_EVENT_FORK will be useful.  Also, it may not be one
process spawning the other new processes.


>
> --
> Sincerely yours,
> Mike.
>

[toc] | [prev] | [next] | [standalone]

#1675929 — Re: [RFC PATCH] userfaultfd: Add feature to request for a signal delivery

On 6/27/17 12:06 AM, Michal Hocko wrote:

> This is an user visible API so let's CC linux-api mailing list.
>
> On Mon 26-06-17 12:46:13, Prakash Sangappa wrote:
>> In some cases, userfaultfd mechanism should just deliver a SIGBUS signal
>> to the faulting process, instead of the page-fault event. Dealing with
>> page-fault event using a monitor thread can be an overhead in these
>> cases. For example applications like the database could use the signaling
>> mechanism for robustness purpose.
> this is rather confusing. What is the reason that the monitor would be
> slower than signal delivery and handling?

There are a large number of single threaded database processes involved,
each of these processes will require a monitor thread which is considered
an overhead.

>
>> Database uses hugetlbfs for performance reason. Files on hugetlbfs
>> filesystem are created and huge pages allocated using fallocate() API.
>> Pages are deallocated/freed using fallocate() hole punching support.
>> These files are mmapped and accessed by many processes as shared memory.
>> The database keeps track of which offsets in the hugetlbfs file have
>> pages allocated.
>>
>> Any access to mapped address over holes in the file, which can occur due
>> to bugs in the application, is considered invalid and expect the process
>> to simply receive a SIGBUS.  However, currently when a hole in the file is
>> accessed via the mapped address, kernel/mm attempts to automatically
>> allocate a page at page fault time, resulting in implicitly filling the
>> hole in the file. This may not be the desired behavior for applications
>> like the database that want to explicitly manage page allocations of
>> hugetlbfs files.
> So you register UFFD_FEATURE_SIGBUS on each region tha you are unmapping
> and than just let those offenders die?

The database application will create the mapping and register with 
userfault.
Subsequently when the processes the mapping over a hole will result in 
SIGBUS
and die.

>
>> Using userfaultfd mechanism, with this support to get a signal, database
>> application can prevent pages from being allocated implicitly when
>> processes access mapped address over holes in the file.
>>
>> This patch adds the feature to request for a SIGBUS signal to userfaultfd
>> mechanism.
>>
>> See following for previous discussion about the database requirement
>> leading to this proposal as suggested by Andrea.
>>
>> http://www.spinics.net/lists/linux-mm/msg129224.html
> Please make those requirements part of the changelog.

The requirement is described above, which is the need for the database
application to not fill hole implicitly. Sorry, if this was not clear. I
will update the change log and send a v2 patch again.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web