Re: [RFC 6/8] nvmet: Be careful about using iomem accesses when dealing with p2pmem

From	Jason Gunthorpe <jgunthorpe@obsidianresearch.com>
Newsgroups	linux.kernel
Subject	Re: [RFC 6/8] nvmet: Be careful about using iomem accesses when dealing with p2pmem
Date	2017-04-04 17:50 +0200
Message-ID	<tsySu-6KL-25@gated-at.bofh.it> (permalink)
References	<tqQA9-4sB-3@gated-at.bofh.it> <tqQAa-4sB-5@gated-at.bofh.it> <tsuvv-43D-17@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

On Tue, Apr 04, 2017 at 01:59:26PM +0300, Sagi Grimberg wrote:
> Note that the nvme completion queues are still on the host memory, so
> this means we have lost the ordering between data and completions as
> they go to different pcie targets.

Hmm, in this simple up/down case with a switch, I think it might
actually be OK.

Transactions might not complete at the NVMe device before the CPU
processes the RDMA completion, however due to the PCI-E ordering rules
new TLPs directed to the NVMe will complete after the RMDA TLPs and
thus observe the new data. (eg order preserving)

It would be very hard to use P2P if fabric ordering is not preserved..

Jason

Thread

[RFC 6/8] nvmet: Be careful about using iomem accesses when dealing with p2pmem Logan Gunthorpe <logang@deltatee.com> - 2017-03-31 00:20 +0200
  Re: [RFC 6/8] nvmet: Be careful about using iomem accesses when  dealing with p2pmem Sagi Grimberg <sagi@grimberg.me> - 2017-04-04 13:10 +0200
    Re: [RFC 6/8] nvmet: Be careful about using iomem accesses when  dealing with p2pmem Jason Gunthorpe <jgunthorpe@obsidianresearch.com> - 2017-04-04 17:50 +0200
      Re: [RFC 6/8] nvmet: Be careful about using iomem accesses when  dealing with p2pmem Logan Gunthorpe <logang@deltatee.com> - 2017-04-04 19:30 +0200
      Re: [RFC 6/8] nvmet: Be careful about using iomem accesses when  dealing with p2pmem Sagi Grimberg <sagi@grimberg.me> - 2017-04-06 07:40 +0200
        Re: [RFC 6/8] nvmet: Be careful about using iomem accesses when  dealing with p2pmem Logan Gunthorpe <logang@deltatee.com> - 2017-04-06 18:10 +0200
        Re: [RFC 6/8] nvmet: Be careful about using iomem accesses when  dealing with p2pmem Jason Gunthorpe <jgunthorpe@obsidianresearch.com> - 2017-04-06 18:40 +0200
        Re: [RFC 6/8] nvmet: Be careful about using iomem accesses when  dealing with p2pmem "Stephen  Bates" <sbates@raithlin.com> - 2017-04-07 13:30 +0200
          Re: [RFC 6/8] nvmet: Be careful about using iomem accesses when  dealing with p2pmem Sagi Grimberg <sagi@grimberg.me> - 2017-04-10 10:40 +0200

csiph-web