Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1722750
| From | Long Li <longli@exchange.microsoft.com> |
|---|---|
| Newsgroups | linux.kernel |
| Subject | [Patch v3 17/19] CIFS: SMBD: Implement SMB read via RDMA write through memory registration |
| Date | 2017-08-29 21:40 +0200 |
| Message-ID | <ujUwG-6Me-39@gated-at.bofh.it> (permalink) |
| References | <ujUwF-6Me-9@gated-at.bofh.it> |
| Organization | linux.* mail to news gateway |
From: Long Li <longli@microsoft.com>
If I/O size is larger than rdma_readwrite_threshold, use RDMA write for
SMB read by specifying channel SMB2_CHANNEL_RDMA_V1 or
SMB2_CHANNEL_RDMA_V1_INVALIDATE in the SMB packet, depending on SMB dialect
used. Append a smbd_buffer_descriptor_v1 to the end of the SMB packet and fill
in other values to indicate this SMB read uses RDMA write.
There is no need to read from the transport for incoming payload. At the time
SMB read response comes back, the data is already transfered and placed in the
pages by RDMA hardware.
Signed-off-by: Long Li <longli@microsoft.com>
---
fs/cifs/file.c | 5 +++++
fs/cifs/smb2pdu.c | 33 +++++++++++++++++++++++++++++++++
2 files changed, 38 insertions(+)
diff --git a/fs/cifs/file.c b/fs/cifs/file.c
index dec70b3..41460a5 100644
--- a/fs/cifs/file.c
+++ b/fs/cifs/file.c
@@ -42,6 +42,7 @@
#include "cifs_debug.h"
#include "cifs_fs_sb.h"
#include "fscache.h"
+#include "smbdirect.h"
static inline int cifs_convert_flags(unsigned int flags)
@@ -3037,6 +3038,8 @@ uncached_fill_pages(struct TCP_Server_Info *server,
}
if (iter)
result = copy_page_from_iter(page, 0, n, iter);
+ else if (rdata->mr)
+ result = n;
else
result = cifs_read_page_from_socket(server, page, n);
if (result < 0)
@@ -3606,6 +3609,8 @@ readpages_fill_pages(struct TCP_Server_Info *server,
if (iter)
result = copy_page_from_iter(page, 0, n, iter);
+ else if (rdata->mr)
+ result = n;
else
result = cifs_read_page_from_socket(server, page, n);
if (result < 0)
diff --git a/fs/cifs/smb2pdu.c b/fs/cifs/smb2pdu.c
index fbad987..1f08c75 100644
--- a/fs/cifs/smb2pdu.c
+++ b/fs/cifs/smb2pdu.c
@@ -2392,6 +2392,39 @@ smb2_new_read_req(void **buf, unsigned int *total_len,
req->Length = cpu_to_le32(io_parms->length);
req->Offset = cpu_to_le64(io_parms->offset);
+ /*
+ * If we want to do a RDMA write, fill in and append
+ * smbd_buffer_descriptor_v1 to the end of read request
+ */
+ if (server->rdma && rdata &&
+ rdata->bytes > server->smbd_conn->rdma_readwrite_threshold) {
+
+ struct smbd_buffer_descriptor_v1 *v1;
+ bool need_invalidate =
+ io_parms->tcon->ses->server->dialect == SMB30_PROT_ID;
+
+ rdata->mr = smbd_register_mr(
+ server->smbd_conn, rdata->pages,
+ rdata->nr_pages, rdata->tailsz,
+ true, need_invalidate);
+ if (!rdata->mr)
+ return -ENOBUFS;
+
+ req->Channel = SMB2_CHANNEL_RDMA_V1_INVALIDATE;
+ if (need_invalidate)
+ req->Channel = SMB2_CHANNEL_RDMA_V1;
+ req->ReadChannelInfoOffset =
+ offsetof(struct smb2_read_plain_req, Buffer);
+ req->ReadChannelInfoLength =
+ sizeof(struct smbd_buffer_descriptor_v1);
+ v1 = (struct smbd_buffer_descriptor_v1 *) &req->Buffer[0];
+ v1->offset = rdata->mr->mr->iova;
+ v1->token = rdata->mr->mr->rkey;
+ v1->length = rdata->mr->mr->length;
+
+ *total_len += sizeof(*v1) - 1;
+ }
+
if (request_type & CHAINED_REQUEST) {
if (!(request_type & END_OF_CHAIN)) {
/* next 8-byte aligned request */
--
2.7.4
Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
[Patch v3 00/19] CIFS: Implement SMB Direct Long Li <longli@exchange.microsoft.com> - 2017-08-29 21:40 +0200
[Patch v3 07/19] CIFS: SMBD: Destroy SMBDirect session on shutdown or umount Long Li <longli@exchange.microsoft.com> - 2017-08-29 21:40 +0200
[Patch v3 04/19] CIFS: SMBD: Add SMBDirect transport to SMB connection and Makefile Long Li <longli@exchange.microsoft.com> - 2017-08-29 21:40 +0200
Re: [Patch v3 04/19] CIFS: SMBD: Add SMBDirect transport to SMB connection and Makefile Pavel Shilovsky <piastryyy@gmail.com> - 2017-08-30 01:20 +0200
RE: [Patch v3 04/19] CIFS: SMBD: Add SMBDirect transport to SMB connection and Makefile Long Li <longli@microsoft.com> - 2017-08-30 01:20 +0200
[Patch v3 09/19] CIFS: SMBD: Receive data from SMBDirect Long Li <longli@exchange.microsoft.com> - 2017-08-29 21:40 +0200
[Patch v3 01/19] CIFS: Add rdma mount option Long Li <longli@exchange.microsoft.com> - 2017-08-29 21:40 +0200
Re: [Patch v3 01/19] CIFS: Add rdma mount option Pavel Shilovsky <piastryyy@gmail.com> - 2017-08-30 00:40 +0200
RE: [Patch v3 01/19] CIFS: Add rdma mount option Long Li <longli@microsoft.com> - 2017-08-30 01:00 +0200
[Patch v3 11/19] CIFS: SMBD: Define memory registration for I/O data Long Li <longli@exchange.microsoft.com> - 2017-08-29 21:40 +0200
Re: [Patch v3 11/19] CIFS: SMBD: Define memory registration for I/O data Pavel Shilovsky <piastryyy@gmail.com> - 2017-08-30 02:50 +0200
RE: [Patch v3 11/19] CIFS: SMBD: Define memory registration for I/O data Long Li <longli@microsoft.com> - 2017-08-30 04:20 +0200
[Patch v3 12/19] CIFS: SMBD: Fix the definition for SMB2_CHANNEL_RDMA_V1_INVALIDATE Long Li <longli@exchange.microsoft.com> - 2017-08-29 21:40 +0200
Re: [Patch v3 12/19] CIFS: SMBD: Fix the definition for SMB2_CHANNEL_RDMA_V1_INVALIDATE Pavel Shilovsky <piastryyy@gmail.com> - 2017-08-30 03:00 +0200
RE: [Patch v3 12/19] CIFS: SMBD: Fix the definition for SMB2_CHANNEL_RDMA_V1_INVALIDATE Long Li <longli@microsoft.com> - 2017-08-30 04:20 +0200
[Patch v3 10/19] CIFS: SMBD: Send data through SMBDirect Long Li <longli@exchange.microsoft.com> - 2017-08-29 21:40 +0200
[Patch v3 19/19] CIFS: SMBD: Add SMBDirect debug counters Long Li <longli@exchange.microsoft.com> - 2017-08-29 21:40 +0200
[Patch v3 17/19] CIFS: SMBD: Implement SMB read via RDMA write through memory registration Long Li <longli@exchange.microsoft.com> - 2017-08-29 21:40 +0200
[Patch v3 14/19] CIFS: SMBD: Deregister memory on finishing SMB write Long Li <longli@exchange.microsoft.com> - 2017-08-29 21:40 +0200
[Patch v3 08/19] CIFS: SMBD: Set SMBDirect maximum read or write size for I/O Long Li <longli@exchange.microsoft.com> - 2017-08-29 21:40 +0200
[Patch v3 18/19] CIFS: SMBD: Deregister memory when finishing SMB read Long Li <longli@exchange.microsoft.com> - 2017-08-29 21:40 +0200
[Patch v3 06/19] CIFS: SMBD: Reconnect to SMBDirect session Long Li <longli@exchange.microsoft.com> - 2017-08-29 21:40 +0200
Re: [Patch v3 06/19] CIFS: SMBD: Reconnect to SMBDirect session Pavel Shilovsky <piastryyy@gmail.com> - 2017-08-30 01:30 +0200
RE: [Patch v3 06/19] CIFS: SMBD: Reconnect to SMBDirect session Long Li <longli@microsoft.com> - 2017-08-30 01:40 +0200
[Patch v3 15/19] CIFS: SMBD: Add parameter rdata to smb2_new_read_req Long Li <longli@exchange.microsoft.com> - 2017-08-29 21:40 +0200
[Patch v3 13/19] CIFS: SMBD: Implement SMB write via RDMA read through memory registration Long Li <longli@exchange.microsoft.com> - 2017-08-29 21:40 +0200
[Patch v3 02/19] CIFS: SMBD: Add SMBDirect protocol and transport constants Long Li <longli@exchange.microsoft.com> - 2017-08-29 21:40 +0200
Re: [Patch v3 02/19] CIFS: SMBD: Add SMBDirect protocol and transport constants Pavel Shilovsky <piastryyy@gmail.com> - 2017-08-30 00:50 +0200
RE: [Patch v3 02/19] CIFS: SMBD: Add SMBDirect protocol and transport constants Long Li <longli@microsoft.com> - 2017-08-30 01:10 +0200
Re: [Patch v3 02/19] CIFS: SMBD: Add SMBDirect protocol and transport constants Pavel Shilovsky <piastryyy@gmail.com> - 2017-08-30 20:20 +0200
RE: [Patch v3 02/19] CIFS: SMBD: Add SMBDirect protocol and transport constants Long Li <longli@microsoft.com> - 2017-08-30 20:30 +0200
Re: [Patch v3 02/19] CIFS: SMBD: Add SMBDirect protocol and transport constants Pavel Shilovsky <piastryyy@gmail.com> - 2017-08-30 20:40 +0200
Re: [Patch v3 02/19] CIFS: SMBD: Add SMBDirect protocol and transport constants Steve French <smfrench@gmail.com> - 2017-09-01 17:50 +0200
RE: [Patch v3 00/19] CIFS: Implement SMB Direct Long Li <longli@microsoft.com> - 2017-08-30 20:50 +0200
csiph-web