Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1330508 > unrolled thread

Another proposal for DAX fault locking

Started byJan Kara <jack@suse.cz>
First post2016-02-09 18:30 +0100
Last post2016-02-11 01:20 +0100
Articles 3 on this page of 23 — 9 participants

Back to article view | Back to linux.kernel

Contents

  Another proposal for DAX fault locking Jan Kara <jack@suse.cz> - 2016-02-09 18:30 +0100
    Re: Another proposal for DAX fault locking Dan Williams <dan.j.williams@intel.com> - 2016-02-09 19:20 +0100
      Re: Another proposal for DAX fault locking Jan Kara <jack@suse.cz> - 2016-02-10 11:40 +0100
        Re: Another proposal for DAX fault locking Dan Williams <dan.j.williams@intel.com> - 2016-02-10 21:10 +0100
          Re: Another proposal for DAX fault locking Jan Kara <jack@suse.cz> - 2016-02-11 11:50 +0100
        Re: Another proposal for DAX fault locking Dave Chinner <david@fromorbit.com> - 2016-02-10 23:20 +0100
          Re: Another proposal for DAX fault locking Cedric Blancher <cedric.blancher@gmail.com> - 2016-02-10 23:40 +0100
            Re: Another proposal for DAX fault locking Ross Zwisler <ross.zwisler@linux.intel.com> - 2016-02-11 00:40 +0100
            Re: Another proposal for DAX fault locking Jan Kara <jack@suse.cz> - 2016-02-11 12:00 +0100
              Re: Another proposal for DAX fault locking Cedric Blancher <cedric.blancher@gmail.com> - 2016-02-11 22:10 +0100
          Re: Another proposal for DAX fault locking Ross Zwisler <ross.zwisler@linux.intel.com> - 2016-02-11 00:40 +0100
            Re: Another proposal for DAX fault locking Jan Kara <jack@suse.cz> - 2016-02-11 12:20 +0100
    Re: Another proposal for DAX fault locking Cedric Blancher <cedric.blancher@gmail.com> - 2016-02-09 19:50 +0100
      Re: Another proposal for DAX fault locking Mel Gorman <mgorman@suse.de> - 2016-02-10 09:20 +0100
        Re: Another proposal for DAX fault locking Jan Kara <jack@suse.cz> - 2016-02-10 11:20 +0100
    Re: Another proposal for DAX fault locking Dmitry Monakhov <dmonlist@gmail.com> - 2016-02-10 13:40 +0100
      Re: Another proposal for DAX fault locking Jan Kara <jack@suse.cz> - 2016-02-10 13:40 +0100
    Re: Another proposal for DAX fault locking Boaz Harrosh <boaz@plexistor.com> - 2016-02-10 18:40 +0100
      Re: Another proposal for DAX fault locking Jan Kara <jack@suse.cz> - 2016-02-11 11:40 +0100
        Re: Another proposal for DAX fault locking Boaz Harrosh <openosd@gmail.com> - 2016-02-14 10:00 +0100
    Re: Another proposal for DAX fault locking Ross Zwisler <ross.zwisler@linux.intel.com> - 2016-02-11 00:50 +0100
      Re: Another proposal for DAX fault locking Cedric Blancher <cedric.blancher@gmail.com> - 2016-02-11 01:00 +0100
        Re: Another proposal for DAX fault locking Ross Zwisler <ross.zwisler@linux.intel.com> - 2016-02-11 01:20 +0100

Page 2 of 2 — ← Prev page 1 [2]

#1331587

FromRoss Zwisler <ross.zwisler@linux.intel.com>
Date2016-02-11 00:50 +0100
Message-ID<r0MGe-ED-3@gated-at.bofh.it>
In reply to#1330508
On Tue, Feb 09, 2016 at 06:24:16PM +0100, Jan Kara wrote:
> Hello,
> 
> I was thinking about current issues with DAX fault locking [1] (data
> corruption due to racing faults allocating blocks) and also races which
> currently don't allow us to clear dirty tags in the radix tree due to races
> between faults and cache flushing [2]. Both of these exist because we don't
> have an equivalent of page lock available for DAX. While we have a
> reasonable solution available for problem [1], so far I'm not aware of a
> decent solution for [2]. After briefly discussing the issue with Mel he had
> a bright idea that we could used hashed locks to deal with [2] (and I think
> we can solve [1] with them as well). So my proposal looks as follows:
> 
> DAX will have an array of mutexes (the array can be made per device but
> initially a global one should be OK). We will use mutexes in the array as a
> replacement for page lock - we will use hashfn(mapping, index) to get
> particular mutex protecting our offset in the mapping. On fault / page
> mkwrite, we'll grab the mutex similarly to page lock and release it once we
> are done updating page tables. This deals with races in [1]. When flushing
> caches we grab the mutex before clearing writeable bit in page tables
> and clearing dirty bit in the radix tree and drop it after we have flushed
> caches for the pfn. This deals with races in [2].
> 
> Thoughts?
> 
> 								Honza
> 
> [1] http://oss.sgi.com/archives/xfs/2016-01/msg00575.html
> [2] https://lists.01.org/pipermail/linux-nvdimm/2016-January/004057.html

Overall I think this sounds promising.  I think a potential tie-in with the
radix tree would maybe take us in a good direction.

I had another idea of how to solve race #2 that involved sticking a seqlock
around the DAX radix tree + pte_mkwrite() sequence, and on the flushing side
if you noticed that you've raced against a page fault, just leaving the dirty
page tree entry intact.

I *think* this could work - I'd want to bang on it more - but if we have a
general way of handling DAX locking that we can use instead of solving these
issues one-by-one as they come up, that seems like a much better route.

[toc] | [prev] | [next] | [standalone]

#1331590

FromCedric Blancher <cedric.blancher@gmail.com>
Date2016-02-11 01:00 +0100
Message-ID<r0MPU-HU-3@gated-at.bofh.it>
In reply to#1331587
There is another "twist" in this game: If there is a huge page with
1GB with a small 4k page as "overlay" (e.g. mmap() MAP_FIXED somewhere
in the middle of a 1GB huge page), hows that handled?

Ced

On 11 February 2016 at 00:44, Ross Zwisler <ross.zwisler@linux.intel.com> wrote:
> On Tue, Feb 09, 2016 at 06:24:16PM +0100, Jan Kara wrote:
>> Hello,
>>
>> I was thinking about current issues with DAX fault locking [1] (data
>> corruption due to racing faults allocating blocks) and also races which
>> currently don't allow us to clear dirty tags in the radix tree due to races
>> between faults and cache flushing [2]. Both of these exist because we don't
>> have an equivalent of page lock available for DAX. While we have a
>> reasonable solution available for problem [1], so far I'm not aware of a
>> decent solution for [2]. After briefly discussing the issue with Mel he had
>> a bright idea that we could used hashed locks to deal with [2] (and I think
>> we can solve [1] with them as well). So my proposal looks as follows:
>>
>> DAX will have an array of mutexes (the array can be made per device but
>> initially a global one should be OK). We will use mutexes in the array as a
>> replacement for page lock - we will use hashfn(mapping, index) to get
>> particular mutex protecting our offset in the mapping. On fault / page
>> mkwrite, we'll grab the mutex similarly to page lock and release it once we
>> are done updating page tables. This deals with races in [1]. When flushing
>> caches we grab the mutex before clearing writeable bit in page tables
>> and clearing dirty bit in the radix tree and drop it after we have flushed
>> caches for the pfn. This deals with races in [2].
>>
>> Thoughts?
>>
>>                                                               Honza
>>
>> [1] http://oss.sgi.com/archives/xfs/2016-01/msg00575.html
>> [2] https://lists.01.org/pipermail/linux-nvdimm/2016-January/004057.html
>
> Overall I think this sounds promising.  I think a potential tie-in with the
> radix tree would maybe take us in a good direction.
>
> I had another idea of how to solve race #2 that involved sticking a seqlock
> around the DAX radix tree + pte_mkwrite() sequence, and on the flushing side
> if you noticed that you've raced against a page fault, just leaving the dirty
> page tree entry intact.
>
> I *think* this could work - I'd want to bang on it more - but if we have a
> general way of handling DAX locking that we can use instead of solving these
> issues one-by-one as they come up, that seems like a much better route.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Cedric Blancher <cedric.blancher@gmail.com>
Institute Pasteur

[toc] | [prev] | [next] | [standalone]

#1331603

FromRoss Zwisler <ross.zwisler@linux.intel.com>
Date2016-02-11 01:20 +0100
Message-ID<r0N9g-13L-21@gated-at.bofh.it>
In reply to#1331590
On Thu, Feb 11, 2016 at 12:51:05AM +0100, Cedric Blancher wrote:
> There is another "twist" in this game: If there is a huge page with
> 1GB with a small 4k page as "overlay" (e.g. mmap() MAP_FIXED somewhere
> in the middle of a 1GB huge page), hows that handled?

Ugh - I'm pretty sure we haven't touched overlays with DAX at all.

The man page says this:

  If the memory region specified by addr and len overlaps pages of any
  existing mapping(s), then the overlapped part of the existing mapping(s)
  will be discarded.

I wonder if this would translate into a hole punch for our DAX mapping,
whatever size it may be, plus an insert?

If so, it seems like we just need to handle each of those operations correctly
on their own (hole punch, insert), and things will take care of themselves?

That being said, I know for a fact that PMD hole punch is currently broken.

> On 11 February 2016 at 00:44, Ross Zwisler <ross.zwisler@linux.intel.com> wrote:
> > On Tue, Feb 09, 2016 at 06:24:16PM +0100, Jan Kara wrote:
> >> Hello,
> >>
> >> I was thinking about current issues with DAX fault locking [1] (data
> >> corruption due to racing faults allocating blocks) and also races which
> >> currently don't allow us to clear dirty tags in the radix tree due to races
> >> between faults and cache flushing [2]. Both of these exist because we don't
> >> have an equivalent of page lock available for DAX. While we have a
> >> reasonable solution available for problem [1], so far I'm not aware of a
> >> decent solution for [2]. After briefly discussing the issue with Mel he had
> >> a bright idea that we could used hashed locks to deal with [2] (and I think
> >> we can solve [1] with them as well). So my proposal looks as follows:
> >>
> >> DAX will have an array of mutexes (the array can be made per device but
> >> initially a global one should be OK). We will use mutexes in the array as a
> >> replacement for page lock - we will use hashfn(mapping, index) to get
> >> particular mutex protecting our offset in the mapping. On fault / page
> >> mkwrite, we'll grab the mutex similarly to page lock and release it once we
> >> are done updating page tables. This deals with races in [1]. When flushing
> >> caches we grab the mutex before clearing writeable bit in page tables
> >> and clearing dirty bit in the radix tree and drop it after we have flushed
> >> caches for the pfn. This deals with races in [2].
> >>
> >> Thoughts?
> >>
> >>                                                               Honza
> >>
> >> [1] http://oss.sgi.com/archives/xfs/2016-01/msg00575.html
> >> [2] https://lists.01.org/pipermail/linux-nvdimm/2016-January/004057.html
> >
> > Overall I think this sounds promising.  I think a potential tie-in with the
> > radix tree would maybe take us in a good direction.
> >
> > I had another idea of how to solve race #2 that involved sticking a seqlock
> > around the DAX radix tree + pte_mkwrite() sequence, and on the flushing side
> > if you noticed that you've raced against a page fault, just leaving the dirty
> > page tree entry intact.
> >
> > I *think* this could work - I'd want to bang on it more - but if we have a
> > general way of handling DAX locking that we can use instead of solving these
> > issues one-by-one as they come up, that seems like a much better route.
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 
> 
> -- 
> Cedric Blancher <cedric.blancher@gmail.com>
> Institute Pasteur

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | linux.kernel

csiph-web