Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1630248 > unrolled thread
| Started by | Johannes Thumshirn <jthumshirn@suse.de> |
|---|---|
| First post | 2017-04-25 09:30 +0200 |
| Last post | 2017-05-01 20:50 +0200 |
| Articles | 2 — 2 participants |
Back to article view | Back to linux.kernel
This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by
below is the oldest one visible, not the original post.
Re: [PATCH] IB/rxe: Don't clamp residual length to mtu Johannes Thumshirn <jthumshirn@suse.de> - 2017-04-25 09:30 +0200
Re: [PATCH] IB/rxe: Don't clamp residual length to mtu Doug Ledford <dledford@redhat.com> - 2017-05-01 20:50 +0200
| From | Johannes Thumshirn <jthumshirn@suse.de> |
|---|---|
| Date | 2017-04-25 09:30 +0200 |
| Subject | Re: [PATCH] IB/rxe: Don't clamp residual length to mtu |
| Message-ID | <tA357-1Bv-1@gated-at.bofh.it> |
On Thu, Apr 06, 2017 at 02:49:44PM +0200, Johannes Thumshirn wrote: > When reading a RDMA WRITE FIRST packet we copy the DMA length from the RDMA > header into the qp->resp.resid variable for later use. Later in check_rkey() > we clamp it to the MTU if the packet is an RDMA WRITE packet and has a > residual length bigger than the MTU. Later in write_data_in() we subtract the > payload of the packet from the residual length. If the packet happens to have a > payload of exactly the MTU size we end up with a residual length of 0 despite > the packet not being the last in the conversation. When the next packet in the > conversation arrives, we don't have any residual length left and thus set the QP > into an error state. > > This broke NVMe over Fabrics functionality over rdma_rxe.ko > > The patch was verified using the following test. > > # echo eth0 > /sys/module/rdma_rxe/parameters/add > # nvme connect -t rdma -a 192.168.155.101 -s 1023 -n nvmf-test > # mkfs.xfs -fK /dev/nvme0n1 > meta-data=/dev/nvme0n1 isize=256 agcount=4, agsize=65536 blks > = sectsz=4096 attr=2, projid32bit=1 > = crc=0 finobt=0, sparse=0 > data = bsize=4096 blocks=262144, imaxpct=25 > = sunit=0 swidth=0 blks > naming =version 2 bsize=4096 ascii-ci=0 ftype=1 > log =internal log bsize=4096 blocks=2560, version=2 > = sectsz=4096 sunit=1 blks, lazy-count=1 > realtime =none extsz=4096 blocks=0, rtextents=0 > # mount /dev/nvme0n1 /tmp/ > [ 148.923263] XFS (nvme0n1): Mounting V4 Filesystem > [ 148.961196] XFS (nvme0n1): Ending clean mount > # dd if=/dev/urandom of=test.bin bs=1M count=128 > 128+0 records in > 128+0 records out > 134217728 bytes (134 MB, 128 MiB) copied, 0.437991 s, 306 MB/s > # sha256sum test.bin > cde42941f045efa8c4f0f157ab6f29741753cdd8d1cff93a6b03649d83c4129a test.bin > # cp test.bin /tmp/ > sha256sum /tmp/test.bin > cde42941f045efa8c4f0f157ab6f29741753cdd8d1cff93a6b03649d83c4129a /tmp/test.bin > > Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de> > Cc: Hannes Reinecke <hare@suse.de> > Cc: Sagi Grimberg <sagi@grimberg.me> > Cc: Max Gurtovoy <maxg@mellanox.com> > --- Doug anything left here? I already have an Ack from Moni. This patch is needed to get NVMe over Fabrics working on rxe so I'd like to see it in v4.12. Thanks, Johannes -- Johannes Thumshirn Storage jthumshirn@suse.de +49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
[toc] | [next] | [standalone]
| From | Doug Ledford <dledford@redhat.com> |
|---|---|
| Date | 2017-05-01 20:50 +0200 |
| Message-ID | <tCoyu-4dM-11@gated-at.bofh.it> |
| In reply to | #1630248 |
On Tue, 2017-04-25 at 09:29 +0200, Johannes Thumshirn wrote: > On Thu, Apr 06, 2017 at 02:49:44PM +0200, Johannes Thumshirn wrote: > > > > When reading a RDMA WRITE FIRST packet we copy the DMA length from > > the RDMA > > header into the qp->resp.resid variable for later use. Later in > > check_rkey() > > we clamp it to the MTU if the packet is an RDMA WRITE packet and > > has a > > residual length bigger than the MTU. Later in write_data_in() we > > subtract the > > payload of the packet from the residual length. If the packet > > happens to have a > > payload of exactly the MTU size we end up with a residual length of > > 0 despite > > the packet not being the last in the conversation. When the next > > packet in the > > conversation arrives, we don't have any residual length left and > > thus set the QP > > into an error state. > > > > This broke NVMe over Fabrics functionality over rdma_rxe.ko > > > > The patch was verified using the following test. > > > > # echo eth0 > /sys/module/rdma_rxe/parameters/add > > # nvme connect -t rdma -a 192.168.155.101 -s 1023 -n nvmf-test > > # mkfs.xfs -fK /dev/nvme0n1 > > meta-data=/dev/nvme0n1 isize=256 agcount=4, > > agsize=65536 blks > > = sectsz=4096 attr=2, > > projid32bit=1 > > = crc=0 finobt=0, sparse=0 > > data = bsize=4096 blocks=262144, > > imaxpct=25 > > = sunit=0 swidth=0 blks > > naming =version 2 bsize=4096 ascii-ci=0 ftype=1 > > log =internal log bsize=4096 blocks=2560, > > version=2 > > = sectsz=4096 sunit=1 blks, lazy- > > count=1 > > realtime =none extsz=4096 blocks=0, > > rtextents=0 > > # mount /dev/nvme0n1 /tmp/ > > [ 148.923263] XFS (nvme0n1): Mounting V4 Filesystem > > [ 148.961196] XFS (nvme0n1): Ending clean mount > > # dd if=/dev/urandom of=test.bin bs=1M count=128 > > 128+0 records in > > 128+0 records out > > 134217728 bytes (134 MB, 128 MiB) copied, 0.437991 s, 306 MB/s > > # sha256sum test.bin > > cde42941f045efa8c4f0f157ab6f29741753cdd8d1cff93a6b03649d83c4129a > > test.bin > > # cp test.bin /tmp/ > > sha256sum /tmp/test.bin > > cde42941f045efa8c4f0f157ab6f29741753cdd8d1cff93a6b03649d83c4129a > > /tmp/test.bin > > > > Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de> > > Cc: Hannes Reinecke <hare@suse.de> > > Cc: Sagi Grimberg <sagi@grimberg.me> > > Cc: Max Gurtovoy <maxg@mellanox.com> > > --- > > Doug anything left here? I already have an Ack from Moni. This patch > is needed > to get NVMe over Fabrics working on rxe so I'd like to see it in > v4.12. Nope, it's all good. I applied it today. -- Doug Ledford <dledford@redhat.com> GPG KeyID: B826A3330E572FDD Key fingerprint = AE6B 1BDA 122B 23B4 265B 1274 B826 A333 0E57 2FDD
[toc] | [prev] | [standalone]
Back to top | Article view | linux.kernel
csiph-web