Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1742571 > unrolled thread
| Started by | "Wei Hu (Xavier)" <xavier.huwei@huawei.com> |
|---|---|
| First post | 2017-09-30 11:10 +0200 |
| Last post | 2017-10-01 07:50 +0200 |
| Articles | 4 — 2 participants |
Back to article view | Back to linux.kernel
[PATCH for-next 0/4] Add Features & Code improvements for hip08 "Wei Hu (Xavier)" <xavier.huwei@huawei.com> - 2017-09-30 11:10 +0200
[PATCH for-next 4/4] RDMA/hns: Update the PD&CQE&MTT specification in hip08 "Wei Hu (Xavier)" <xavier.huwei@huawei.com> - 2017-09-30 11:10 +0200
[PATCH for-next 3/4] RDMA/hns: Update the IRRL table chunk size in hip08 "Wei Hu (Xavier)" <xavier.huwei@huawei.com> - 2017-09-30 11:10 +0200
Re: [PATCH for-next 3/4] RDMA/hns: Update the IRRL table chunk size in hip08 Leon Romanovsky <leon@kernel.org> - 2017-10-01 07:50 +0200
| From | "Wei Hu (Xavier)" <xavier.huwei@huawei.com> |
|---|---|
| Date | 2017-09-30 11:10 +0200 |
| Subject | [PATCH for-next 0/4] Add Features & Code improvements for hip08 |
| Message-ID | <uvlWy-28j-3@gated-at.bofh.it> |
This patch-set introduce PBL page size configuration support,IOMMU support, updating PD&CQE&MTT specification and IRRL table chunk size for hip08. Shaobo Xu (1): RDMA/hns: Support WQE/CQE/PBL page size configurable feature in hip08 Wei Hu (Xavier) (3): RDMA/hns: Add IOMMU enable support in hip08 RDMA/hns: Update the IRRL table chunk size in hip08 RDMA/hns: Update the PD&CQE&MTT specification in hip08 drivers/infiniband/hw/hns/hns_roce_alloc.c | 34 +++++++---- drivers/infiniband/hw/hns/hns_roce_cq.c | 21 ++++++- drivers/infiniband/hw/hns/hns_roce_device.h | 13 ++-- drivers/infiniband/hw/hns/hns_roce_hem.c | 61 +++++++++++++------ drivers/infiniband/hw/hns/hns_roce_hem.h | 6 ++ drivers/infiniband/hw/hns/hns_roce_hw_v1.c | 1 + drivers/infiniband/hw/hns/hns_roce_hw_v1.h | 2 + drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 23 ++++--- drivers/infiniband/hw/hns/hns_roce_hw_v2.h | 10 ++-- drivers/infiniband/hw/hns/hns_roce_mr.c | 93 ++++++++++++++++++++--------- drivers/infiniband/hw/hns/hns_roce_qp.c | 46 ++++++++++---- 11 files changed, 222 insertions(+), 88 deletions(-) -- 1.9.1
[toc] | [next] | [standalone]
| From | "Wei Hu (Xavier)" <xavier.huwei@huawei.com> |
|---|---|
| Date | 2017-09-30 11:10 +0200 |
| Subject | [PATCH for-next 4/4] RDMA/hns: Update the PD&CQE&MTT specification in hip08 |
| Message-ID | <uvlWy-28j-11@gated-at.bofh.it> |
| In reply to | #1742571 |
This patch updates the PD specification to 16M for hip08. And it updates the numbers of mtt and cqe segments for the buddy. As the CQE supports hop num 1 addressing, the CQE specification is 64k. This patch updates to set the CQE specification to 64k. Signed-off-by: Shaobo Xu <xushaobo2@huawei.com> Signed-off-by: Wei Hu (Xavier) <xavier.huwei@huawei.com> Signed-off-by: Lijun Ou <oulijun@huawei.com> --- drivers/infiniband/hw/hns/hns_roce_hw_v2.h | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.h b/drivers/infiniband/hw/hns/hns_roce_hw_v2.h index 65ed3f8..6106ad1 100644 --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.h +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.h @@ -47,16 +47,16 @@ #define HNS_ROCE_V2_MAX_QP_NUM 0x2000 #define HNS_ROCE_V2_MAX_WQE_NUM 0x8000 #define HNS_ROCE_V2_MAX_CQ_NUM 0x8000 -#define HNS_ROCE_V2_MAX_CQE_NUM 0x400000 +#define HNS_ROCE_V2_MAX_CQE_NUM 0x10000 #define HNS_ROCE_V2_MAX_RQ_SGE_NUM 0x100 #define HNS_ROCE_V2_MAX_SQ_SGE_NUM 0xff #define HNS_ROCE_V2_MAX_SQ_INLINE 0x20 #define HNS_ROCE_V2_UAR_NUM 256 #define HNS_ROCE_V2_PHY_UAR_NUM 1 #define HNS_ROCE_V2_MAX_MTPT_NUM 0x8000 -#define HNS_ROCE_V2_MAX_MTT_SEGS 0x100000 -#define HNS_ROCE_V2_MAX_CQE_SEGS 0x10000 -#define HNS_ROCE_V2_MAX_PD_NUM 0x400000 +#define HNS_ROCE_V2_MAX_MTT_SEGS 0x1000000 +#define HNS_ROCE_V2_MAX_CQE_SEGS 0x1000000 +#define HNS_ROCE_V2_MAX_PD_NUM 0x1000000 #define HNS_ROCE_V2_MAX_QP_INIT_RDMA 128 #define HNS_ROCE_V2_MAX_QP_DEST_RDMA 128 #define HNS_ROCE_V2_MAX_SQ_DESC_SZ 64 -- 1.9.1
[toc] | [prev] | [next] | [standalone]
| From | "Wei Hu (Xavier)" <xavier.huwei@huawei.com> |
|---|---|
| Date | 2017-09-30 11:10 +0200 |
| Subject | [PATCH for-next 3/4] RDMA/hns: Update the IRRL table chunk size in hip08 |
| Message-ID | <uvlWy-28j-13@gated-at.bofh.it> |
| In reply to | #1742571 |
As the increase of the IRRL specification in hip08, the IRRL table
chunk size needs to be updated.
This patch updates the IRRL table chunk size to 256k for hip08.
Signed-off-by: Wei Hu (Xavier) <xavier.huwei@huawei.com>
Signed-off-by: Shaobo Xu <xushaobo2@huawei.com>
Signed-off-by: Lijun Ou <oulijun@huawei.com>
---
drivers/infiniband/hw/hns/hns_roce_device.h | 3 +++
drivers/infiniband/hw/hns/hns_roce_hem.c | 31 ++++++++++++++---------------
drivers/infiniband/hw/hns/hns_roce_hw_v1.c | 1 +
drivers/infiniband/hw/hns/hns_roce_hw_v1.h | 2 ++
drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 1 +
drivers/infiniband/hw/hns/hns_roce_hw_v2.h | 2 ++
6 files changed, 24 insertions(+), 16 deletions(-)
diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h
index 9353400..fc2a53d 100644
--- a/drivers/infiniband/hw/hns/hns_roce_device.h
+++ b/drivers/infiniband/hw/hns/hns_roce_device.h
@@ -236,6 +236,8 @@ struct hns_roce_hem_table {
unsigned long num_obj;
/*Single obj size */
unsigned long obj_size;
+ unsigned long table_chunk_size;
+ unsigned long hem_alloc_size;
int lowmem;
struct mutex mutex;
struct hns_roce_hem **hem;
@@ -565,6 +567,7 @@ struct hns_roce_caps {
u32 cqe_ba_pg_sz;
u32 cqe_buf_pg_sz;
u32 cqe_hop_num;
+ u32 chunk_sz; /* chunk size in non multihop mode*/
};
struct hns_roce_hw {
diff --git a/drivers/infiniband/hw/hns/hns_roce_hem.c b/drivers/infiniband/hw/hns/hns_roce_hem.c
index 4a3d1d4..c08bc16 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hem.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hem.c
@@ -36,9 +36,6 @@
#include "hns_roce_hem.h"
#include "hns_roce_common.h"
-#define HNS_ROCE_HEM_ALLOC_SIZE (1 << 17)
-#define HNS_ROCE_TABLE_CHUNK_SIZE (1 << 17)
-
#define DMA_ADDR_T_SHIFT 12
#define BT_BA_SHIFT 32
@@ -314,7 +311,7 @@ static int hns_roce_set_hem(struct hns_roce_dev *hr_dev,
/* Find the HEM(Hardware Entry Memory) entry */
unsigned long i = (obj & (table->num_obj - 1)) /
- (HNS_ROCE_TABLE_CHUNK_SIZE / table->obj_size);
+ (table->table_chunk_size / table->obj_size);
switch (table->type) {
case HEM_TYPE_QPC:
@@ -559,7 +556,7 @@ int hns_roce_table_get(struct hns_roce_dev *hr_dev,
if (hns_roce_check_whether_mhop(hr_dev, table->type))
return hns_roce_table_mhop_get(hr_dev, table, obj);
- i = (obj & (table->num_obj - 1)) / (HNS_ROCE_TABLE_CHUNK_SIZE /
+ i = (obj & (table->num_obj - 1)) / (table->table_chunk_size /
table->obj_size);
mutex_lock(&table->mutex);
@@ -570,8 +567,8 @@ int hns_roce_table_get(struct hns_roce_dev *hr_dev,
}
table->hem[i] = hns_roce_alloc_hem(hr_dev,
- HNS_ROCE_TABLE_CHUNK_SIZE >> PAGE_SHIFT,
- HNS_ROCE_HEM_ALLOC_SIZE,
+ table->table_chunk_size >> PAGE_SHIFT,
+ table->hem_alloc_size,
(table->lowmem ? GFP_KERNEL :
GFP_HIGHUSER) | __GFP_NOWARN);
if (!table->hem[i]) {
@@ -720,7 +717,7 @@ void hns_roce_table_put(struct hns_roce_dev *hr_dev,
}
i = (obj & (table->num_obj - 1)) /
- (HNS_ROCE_TABLE_CHUNK_SIZE / table->obj_size);
+ (table->table_chunk_size / table->obj_size);
mutex_lock(&table->mutex);
@@ -757,8 +754,8 @@ void *hns_roce_table_find(struct hns_roce_dev *hr_dev,
if (!hns_roce_check_whether_mhop(hr_dev, table->type)) {
idx = (obj & (table->num_obj - 1)) * table->obj_size;
- hem = table->hem[idx / HNS_ROCE_TABLE_CHUNK_SIZE];
- dma_offset = offset = idx % HNS_ROCE_TABLE_CHUNK_SIZE;
+ hem = table->hem[idx / table->table_chunk_size];
+ dma_offset = offset = idx % table->table_chunk_size;
} else {
hns_roce_calc_hem_mhop(hr_dev, table, &mhop_obj, &mhop);
/* mtt mhop */
@@ -815,7 +812,7 @@ int hns_roce_table_get_range(struct hns_roce_dev *hr_dev,
unsigned long start, unsigned long end)
{
struct hns_roce_hem_mhop mhop;
- unsigned long inc = HNS_ROCE_TABLE_CHUNK_SIZE / table->obj_size;
+ unsigned long inc = table->table_chunk_size / table->obj_size;
unsigned long i;
int ret;
@@ -846,7 +843,7 @@ void hns_roce_table_put_range(struct hns_roce_dev *hr_dev,
unsigned long start, unsigned long end)
{
struct hns_roce_hem_mhop mhop;
- unsigned long inc = HNS_ROCE_TABLE_CHUNK_SIZE / table->obj_size;
+ unsigned long inc = table->table_chunk_size / table->obj_size;
unsigned long i;
if (hns_roce_check_whether_mhop(hr_dev, table->type)) {
@@ -854,8 +851,7 @@ void hns_roce_table_put_range(struct hns_roce_dev *hr_dev,
inc = mhop.bt_chunk_size / table->obj_size;
}
- for (i = start; i <= end;
- i += inc)
+ for (i = start; i <= end; i += inc)
hns_roce_table_put(hr_dev, table, i);
}
@@ -869,7 +865,10 @@ int hns_roce_init_hem_table(struct hns_roce_dev *hr_dev,
unsigned long num_hem;
if (!hns_roce_check_whether_mhop(hr_dev, type)) {
- obj_per_chunk = HNS_ROCE_TABLE_CHUNK_SIZE / obj_size;
+ table->table_chunk_size = hr_dev->caps.chunk_sz;
+ table->hem_alloc_size = hr_dev->caps.chunk_sz;
+
+ obj_per_chunk = table->table_chunk_size / obj_size;
num_hem = (nobj + obj_per_chunk - 1) / obj_per_chunk;
table->hem = kcalloc(num_hem, sizeof(*table->hem), GFP_KERNEL);
@@ -1051,7 +1050,7 @@ void hns_roce_cleanup_hem_table(struct hns_roce_dev *hr_dev,
for (i = 0; i < table->num_hem; ++i)
if (table->hem[i]) {
if (hr_dev->hw->clear_hem(hr_dev, table,
- i * HNS_ROCE_TABLE_CHUNK_SIZE / table->obj_size, 0))
+ i * table->table_chunk_size / table->obj_size, 0))
dev_err(dev, "Clear HEM base address failed.\n");
hns_roce_free_hem(hr_dev, table->hem[i]);
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v1.c b/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
index 852db18..47ff1c9 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
@@ -1513,6 +1513,7 @@ int hns_roce_v1_profile(struct hns_roce_dev *hr_dev)
caps->reserved_mrws = 1;
caps->reserved_uars = 0;
caps->reserved_cqs = 0;
+ caps->chunk_sz = HNS_ROCE_V1_TABLE_CHUNK_SIZE;
for (i = 0; i < caps->num_ports; i++)
caps->pkey_table_len[i] = 1;
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v1.h b/drivers/infiniband/hw/hns/hns_roce_hw_v1.h
index eb83ff3..21a07ef 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v1.h
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v1.h
@@ -72,6 +72,8 @@
#define HNS_ROCE_V1_CQE_ENTRY_SIZE 32
#define HNS_ROCE_V1_PAGE_SIZE_SUPPORT 0xFFFFF000
+#define HNS_ROCE_V1_TABLE_CHUNK_SIZE (1 << 17)
+
#define HNS_ROCE_V1_EXT_RAQ_WF 8
#define HNS_ROCE_V1_RAQ_ENTRY 64
#define HNS_ROCE_V1_RAQ_DEPTH 32768
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
index 9e19bf1..5a011da 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
@@ -943,6 +943,7 @@ static int hns_roce_v2_profile(struct hns_roce_dev *hr_dev)
caps->cqe_ba_pg_sz = 0;
caps->cqe_buf_pg_sz = 0;
caps->cqe_hop_num = HNS_ROCE_CQE_HOP_NUM;
+ caps->chunk_sz = HNS_ROCE_V2_TABLE_CHUNK_SIZE;
caps->pkey_table_len[0] = 1;
caps->gid_table_len[0] = 2;
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.h b/drivers/infiniband/hw/hns/hns_roce_hw_v2.h
index 4fc4acd..65ed3f8 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.h
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.h
@@ -78,6 +78,8 @@
#define HNS_ROCE_CQE_HOP_NUM 1
#define HNS_ROCE_PBL_HOP_NUM 2
+#define HNS_ROCE_V2_TABLE_CHUNK_SIZE (1 << 18)
+
#define HNS_ROCE_CMD_FLAG_IN_VALID_SHIFT 0
#define HNS_ROCE_CMD_FLAG_OUT_VALID_SHIFT 1
#define HNS_ROCE_CMD_FLAG_NEXT_SHIFT 2
--
1.9.1
[toc] | [prev] | [next] | [standalone]
| From | Leon Romanovsky <leon@kernel.org> |
|---|---|
| Date | 2017-10-01 07:50 +0200 |
| Subject | Re: [PATCH for-next 3/4] RDMA/hns: Update the IRRL table chunk size in hip08 |
| Message-ID | <uvFix-6rC-1@gated-at.bofh.it> |
| In reply to | #1742573 |
[Multipart message — attachments visible in raw view] — view raw
On Sat, Sep 30, 2017 at 05:29:00PM +0800, Wei Hu (Xavier) wrote:
> As the increase of the IRRL specification in hip08, the IRRL table
> chunk size needs to be updated.
> This patch updates the IRRL table chunk size to 256k for hip08.
>
> Signed-off-by: Wei Hu (Xavier) <xavier.huwei@huawei.com>
> Signed-off-by: Shaobo Xu <xushaobo2@huawei.com>
> Signed-off-by: Lijun Ou <oulijun@huawei.com>
> ---
> drivers/infiniband/hw/hns/hns_roce_device.h | 3 +++
> drivers/infiniband/hw/hns/hns_roce_hem.c | 31 ++++++++++++++---------------
> drivers/infiniband/hw/hns/hns_roce_hw_v1.c | 1 +
> drivers/infiniband/hw/hns/hns_roce_hw_v1.h | 2 ++
> drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 1 +
> drivers/infiniband/hw/hns/hns_roce_hw_v2.h | 2 ++
> 6 files changed, 24 insertions(+), 16 deletions(-)
>
> diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h
> index 9353400..fc2a53d 100644
> --- a/drivers/infiniband/hw/hns/hns_roce_device.h
> +++ b/drivers/infiniband/hw/hns/hns_roce_device.h
> @@ -236,6 +236,8 @@ struct hns_roce_hem_table {
> unsigned long num_obj;
> /*Single obj size */
> unsigned long obj_size;
> + unsigned long table_chunk_size;
> + unsigned long hem_alloc_size;
> int lowmem;
> struct mutex mutex;
> struct hns_roce_hem **hem;
> @@ -565,6 +567,7 @@ struct hns_roce_caps {
> u32 cqe_ba_pg_sz;
> u32 cqe_buf_pg_sz;
> u32 cqe_hop_num;
> + u32 chunk_sz; /* chunk size in non multihop mode*/
> };
Hi,
I have two comments:
1. In this code table_chunk_size is equal and similar to hem_alloc_size.
Please don't introduce unneeded complexity.
2. The size of table is num_obj * obj_size, there is no need to
table_chunk_size and hem_alloc_size at all. There are plenty of macros in
the kernel to deal with the tables.
Thanks
[toc] | [prev] | [standalone]
Back to top | Article view | linux.kernel
csiph-web