Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1390542 > unrolled thread

[PATCH v4 5/7] fs: prioritize and separate direct_io from dax_io

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  [PATCH v4 5/7] fs: prioritize and separate direct_io from dax_io Vishal Verma <vishal.l.verma@intel.com> - 2016-04-28 23:20 +0200
    Re: [PATCH v4 5/7] fs: prioritize and separate direct_io from dax_io Christoph Hellwig <hch@infradead.org> - 2016-05-02 17:00 +0200
      Re: [PATCH v4 5/7] fs: prioritize and separate direct_io from dax_io Vishal Verma <vishal@kernel.org> - 2016-05-02 17:50 +0200
    Re: [PATCH v4 5/7] fs: prioritize and separate direct_io from dax_io Boaz Harrosh <boaz@plexistor.com> - 2016-05-02 17:50 +0200
      Re: [PATCH v4 5/7] fs: prioritize and separate direct_io from dax_io Vishal Verma <vishal@kernel.org> - 2016-05-02 18:00 +0200
        Re: [PATCH v4 5/7] fs: prioritize and separate direct_io from dax_io Boaz Harrosh <boaz@plexistor.com> - 2016-05-02 18:10 +0200
          Re: [PATCH v4 5/7] fs: prioritize and separate direct_io from dax_io "Verma, Vishal L" <vishal.l.verma@intel.com> - 2016-05-02 21:00 +0200
      Re: [PATCH v4 5/7] fs: prioritize and separate direct_io from dax_io Dan Williams <dan.j.williams@intel.com> - 2016-05-02 18:10 +0200
        Re: [PATCH v4 5/7] fs: prioritize and separate direct_io from dax_io Boaz Harrosh <boaz@plexistor.com> - 2016-05-02 18:30 +0200
          Re: [PATCH v4 5/7] fs: prioritize and separate direct_io from dax_io Dan Williams <dan.j.williams@intel.com> - 2016-05-02 19:00 +0200
            Re: [PATCH v4 5/7] fs: prioritize and separate direct_io from dax_io Boaz Harrosh <boaz@plexistor.com> - 2016-05-02 19:50 +0200
              Re: [PATCH v4 5/7] fs: prioritize and separate direct_io from dax_io Dan Williams <dan.j.williams@intel.com> - 2016-05-02 20:20 +0200
                Re: [PATCH v4 5/7] fs: prioritize and separate direct_io from dax_io Boaz Harrosh <boaz@plexistor.com> - 2016-05-02 20:40 +0200
                  Re: [PATCH v4 5/7] fs: prioritize and separate direct_io from dax_io Dan Williams <dan.j.williams@intel.com> - 2016-05-02 20:50 +0200
                    Re: [PATCH v4 5/7] fs: prioritize and separate direct_io from dax_io Boaz Harrosh <boaz@plexistor.com> - 2016-05-02 21:30 +0200
      Re: [PATCH v4 5/7] fs: prioritize and separate direct_io from dax_io Christoph Hellwig <hch@infradead.org> - 2016-05-05 16:30 +0200
        Re: [PATCH v4 5/7] fs: prioritize and separate direct_io from dax_io Dan Williams <dan.j.williams@intel.com> - 2016-05-05 17:20 +0200
          Re: [PATCH v4 5/7] fs: prioritize and separate direct_io from dax_io Christoph Hellwig <hch@infradead.org> - 2016-05-05 17:30 +0200
            Re: [PATCH v4 5/7] fs: prioritize and separate direct_io from dax_io Dan Williams <dan.j.williams@intel.com> - 2016-05-05 18:30 +0200
            Re: [PATCH v4 5/7] fs: prioritize and separate direct_io from dax_io "Verma, Vishal L" <vishal.l.verma@intel.com> - 2016-05-05 23:50 +0200
              Re: [PATCH v4 5/7] fs: prioritize and separate direct_io from dax_io "hch@infradead.org" <hch@infradead.org> - 2016-05-08 11:10 +0200
                Re: [PATCH v4 5/7] fs: prioritize and separate direct_io from dax_io "Verma, Vishal L" <vishal.l.verma@intel.com> - 2016-05-08 20:50 +0200
          Re: [PATCH v4 5/7] fs: prioritize and separate direct_io from dax_io "Verma, Vishal L" <vishal.l.verma@intel.com> - 2016-05-05 23:50 +0200
        Re: [PATCH v4 5/7] fs: prioritize and separate direct_io from dax_io "Verma, Vishal L" <vishal.l.verma@intel.com> - 2016-05-05 23:40 +0200
          Re: [PATCH v4 5/7] fs: prioritize and separate direct_io from dax_io "hch@infradead.org" <hch@infradead.org> - 2016-05-08 11:10 +0200

Page 2 of 2 — ← Prev page 1 [2]

#1396399

On Thu, May 05, 2016 at 09:45:07PM +0000, Verma, Vishal L wrote:
> I'm not sure I completely understand how this will work? Can you explain
> a bit? Would we have to export rw_bytes up to layers above the pmem
> driver? Where does get_user_pages come in?

A DAX filesystem can directly use the nvdimm layer the same way btt
doe,s what's the problem?

Re get_user_pages my idea was to simply use that to lock down the user
pages so that we can call rw_bytes on it.  How else would you do it?  Do
a kmalloc, copy_from_user and then another memcpy?

[toc] | [prev] | [next] | [standalone]

#1396450

On Sun, 2016-05-08 at 02:01 -0700, hch@infradead.org wrote:
> On Thu, May 05, 2016 at 09:45:07PM +0000, Verma, Vishal L wrote:
> > 
> > I'm not sure I completely understand how this will work? Can you
> > explain
> > a bit? Would we have to export rw_bytes up to layers above the pmem
> > driver? Where does get_user_pages come in?
> A DAX filesystem can directly use the nvdimm layer the same way btt
> doe,s what's the problem?

The BTT does rw_bytes through an internal-to-libnvdimm mechanism, but
rw_bytes isn't exported to the filesystem, currently.. To do this we'd
have to either add an rw_bytes to block device operations...or
something.

Another thing is rw_bytes currently doesn't do error clearing either.
We store badblocks at sector granularity, and like Dan said earlier,
that hides the clear_error alignment requirements and upper layers
don't have to be aware of it. To make rw_bytes clear sub-sector errors,
we'd have to change the granularity of bad-blocks, and make upper
layers aware of the clearing alignment requirements.

Using a block-write semantic for clearing hides all this away.

> 
> Re get_user_pages my idea was to simply use that to lock down the
> user
> pages so that we can call rw_bytes on it.  How else would you do
> it?  Do
> a kmalloc, copy_from_user and then another memcpy?

[toc] | [prev] | [next] | [standalone]

#1395388

On Thu, 2016-05-05 at 08:15 -0700, Dan Williams wrote:
> On Thu, May 5, 2016 at 7:24 AM, Christoph Hellwig <hch@infradead.org>
> wrote:
> > 
> > On Mon, May 02, 2016 at 06:41:51PM +0300, Boaz Harrosh wrote:
> > > 
> > > > 
> > > > All IO in a dax filesystem used to go through dax_do_io, which
> > > > cannot
> > > > handle media errors, and thus cannot provide a recovery path
> > > > that can
> > > > send a write through the driver to clear errors.
> > > > 
> > > > Add a new iocb flag for DAX, and set it only for DAX mounts. In
> > > > the IO
> > > > path for DAX filesystems, use the same direct_IO path for both
> > > > DAX and
> > > > direct_io iocbs, but use the flags to identify when we are in
> > > > O_DIRECT
> > > > mode vs non O_DIRECT with DAX, and for O_DIRECT, use the
> > > > conventional
> > > > direct_IO path instead of DAX.
> > > > 
> > > Really? What are your thinking here?
> > > 
> > > What about all the current users of O_DIRECT, you have just made
> > > them
> > > 4 times slower and "less concurrent*" then "buffred io" users.
> > > Since
> > > direct_IO path will queue an IO request and all.
> > > (And if it is not so slow then why do we need dax_do_io at all?
> > > [Rhetorical])
> > > 
> > > I hate it that you overload the semantics of a known and expected
> > > O_DIRECT flag, for special pmem quirks. This is an incompatible
> > > and unrelated overload of the semantics of O_DIRECT.
> > Agreed - makig O_DIRECT less direct than not having it is plain
> > stupid,
> > and I somehow missed this initially.
> Of course I disagree because like Dave argues in the msync case we
> should do the correct thing first and make it fast later, but also
> like Dave this arguing in circles is getting tiresome.
> 
> > 
> > This whole DAX story turns into a major nightmare, and I fear all
> > our
> > hodge podge tweaks to the semantics aren't helping it.
> > 
> > It seems like we simply need an explicit O_DAX for the read/write
> > bypass if can't sort out the semantics (error, writer
> > synchronization)
> > just as we need a special flag for MMAP.
> I don't see how O_DAX makes this situation better if the goal is to
> accelerate unmodified applications...
> 
> Vishal, at least the "delete a file with a badblock" model will still
> work for implicitly clearing errors with your changes to stop doing
> block clearing in fs/dax.c.  This combined with a new -EBADBLOCK (as
> Dave suggests) and explicit logging of I/Os that fail for this reason
> at least gives a chance to communicate errors in files to suitably
> aware applications / environments.

Agreed - I'll send out a series that has just the zeroing changes, and
drop the dax_io fallback/O_DIRECT tweak for now while we figure out the
right thing to do. That should get us to a place where we still have dax
in the presence of errors, and have _a_ path for recovery.

> _______________________________________________
> Linux-nvdimm mailing list
> Linux-nvdimm@lists.01.org
> https://lists.01.org/mailman/listinfo/linux-nvdimm

[toc] | [prev] | [next] | [standalone]

#1395381

On Thu, 2016-05-05 at 07:24 -0700, Christoph Hellwig wrote:
> On Mon, May 02, 2016 at 06:41:51PM +0300, Boaz Harrosh wrote:
> > 
> > > 
> > > All IO in a dax filesystem used to go through dax_do_io, which
> > > cannot
> > > handle media errors, and thus cannot provide a recovery path that
> > > can
> > > send a write through the driver to clear errors.
> > > 
> > > Add a new iocb flag for DAX, and set it only for DAX mounts. In
> > > the IO
> > > path for DAX filesystems, use the same direct_IO path for both DAX
> > > and
> > > direct_io iocbs, but use the flags to identify when we are in
> > > O_DIRECT
> > > mode vs non O_DIRECT with DAX, and for O_DIRECT, use the
> > > conventional
> > > direct_IO path instead of DAX.
> > > 
> > Really? What are your thinking here?
> > 
> > What about all the current users of O_DIRECT, you have just made
> > them
> > 4 times slower and "less concurrent*" then "buffred io" users. Since
> > direct_IO path will queue an IO request and all.
> > (And if it is not so slow then why do we need dax_do_io at all?
> > [Rhetorical])
> > 
> > I hate it that you overload the semantics of a known and expected
> > O_DIRECT flag, for special pmem quirks. This is an incompatible
> > and unrelated overload of the semantics of O_DIRECT.
> Agreed - makig O_DIRECT less direct than not having it is plain
> stupid,
> and I somehow missed this initially.

How is it any 'less direct'? All it does now is follow the blockdev
O_DIRECT path. There still isn't any page cache involved..

> 
> This whole DAX story turns into a major nightmare, and I fear all our
> hodge podge tweaks to the semantics aren't helping it.
> 
> It seems like we simply need an explicit O_DAX for the read/write
> bypass if can't sort out the semantics (error, writer synchronization)
> just as we need a special flag for MMAP..

[toc] | [prev] | [next] | [standalone]

#1396400

On Thu, May 05, 2016 at 09:39:14PM +0000, Verma, Vishal L wrote:
> How is it any 'less direct'? All it does now is follow the blockdev
> O_DIRECT path. There still isn't any page cache involved..

It's still more overhead than the play DAX I/O path.

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | linux.kernel

csiph-web