RE: [PATCH 06/16] ntb: add check and comment for link up to mw_count and mw_get_align

From	"Hubbe, Allen" <Allen.Hubbe@dell.com>
Newsgroups	linux.kernel
Subject	RE: [PATCH 06/16] ntb: add check and comment for link up to mw_count and mw_get_align
Date	2017-06-29 23:30 +0200
Message-ID	<tXPaH-4Ds-29@gated-at.bofh.it> (permalink)
References	(1 earlier) <tXyjx-2hu-15@gated-at.bofh.it> <tXMcO-2zD-15@gated-at.bofh.it> <tXMZc-354-15@gated-at.bofh.it> <tXO4W-3Qo-9@gated-at.bofh.it> <tXO4W-3Qo-7@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

From: Logan Gunthorpe
> On 6/29/2017 2:13 PM, Allen Hubbe wrote:
> > Unfortunately, it is to work around hardware errata.  That is not so trivial to fix.
> 
> Can you describe more what the work around is doing? Can you share the
> code? It seems odd that a workaround is based on the alignment
> restrictions of the mws.

Sure, while not making any claim that this is ready for upstream.

It is not a workaround for alignment restrictions of the mws.  It is a restriction to avoid the use of doorbells and scratchpads.  Memory windows are used exclusively.

Read msi-x local <addr,data> and send that to the peer:
https://github.com/ntrdma/ntrdma/blob/master/drivers/ntc/ntc_ntb_msi.c#L603

Transform peer's addr to the memory window region:
https://github.com/ntrdma/ntrdma/blob/master/drivers/ntc/ntc_ntb_msi.c#L622

Append a dma immediate value operation after other operations, to write the data at addr:
https://github.com/ntrdma/ntrdma/blob/master/drivers/ntc/ntc_ntb_msi.c#L1208

Above describes the workaround.

Used in the context of a rdma-over-ntb driver here:
https://github.com/ntrdma/ntrdma/blob/master/drivers/infiniband/hw/ntrdma/ntrdma_qp.c#L1585

> 
> Logan

Thread

[PATCH 06/16] ntb: add check and comment for link up to mw_count and mw_get_align Logan Gunthorpe <logang@deltatee.com> - 2017-06-29 05:30 +0200
  RE: [PATCH 06/16] ntb: add check and comment for link up to mw_count and mw_get_align "Allen Hubbe" <Allen.Hubbe@dell.com> - 2017-06-29 20:20 +0200
    Re: [PATCH 06/16] ntb: add check and comment for link up to mw_count  and mw_get_align Logan Gunthorpe <logang@deltatee.com> - 2017-06-29 21:10 +0200
      Re: [PATCH 06/16] ntb: add check and comment for link up to mw_count  and mw_get_align Logan Gunthorpe <logang@deltatee.com> - 2017-06-29 22:20 +0200
        RE: [PATCH 06/16] ntb: add check and comment for link up to  mw_count and mw_get_align "Hubbe, Allen" <Allen.Hubbe@dell.com> - 2017-06-29 23:30 +0200
        RE: [PATCH 06/16] ntb: add check and comment for link up to mw_count and mw_get_align "Allen Hubbe" <Allen.Hubbe@dell.com> - 2017-06-29 23:40 +0200
          Re: [PATCH 06/16] ntb: add check and comment for link up to mw_count  and mw_get_align Logan Gunthorpe <logang@deltatee.com> - 2017-06-30 00:20 +0200
      RE: [PATCH 06/16] ntb: add check and comment for link up to mw_count and mw_get_align "Allen Hubbe" <Allen.Hubbe@dell.com> - 2017-06-29 22:20 +0200

csiph-web