Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > linux.kernel > #1702870 > unrolled thread

[PATCH V3 net-next 00/21] Huawei HiNIC Ethernet Driver

Started byAviad Krawczyk <aviad.krawczyk@huawei.com>
First post2017-08-03 12:00 +0200
Last post2017-08-08 19:10 +0200
Articles 12 — 2 participants

Back to article view | Back to linux.kernel


Contents

  [PATCH V3 net-next 00/21] Huawei HiNIC Ethernet Driver Aviad Krawczyk <aviad.krawczyk@huawei.com> - 2017-08-03 12:00 +0200
    [PATCH V3 net-next 15/21] net-next/hinic: Add ceqs Aviad Krawczyk <aviad.krawczyk@huawei.com> - 2017-08-03 12:00 +0200
    [PATCH V3 net-next 04/21] net-next/hinic: Initialize api cmd hw Aviad Krawczyk <aviad.krawczyk@huawei.com> - 2017-08-03 12:00 +0200
    [PATCH V3 net-next 20/21] net-next/hinic: Add ethtool and stats Aviad Krawczyk <aviad.krawczyk@huawei.com> - 2017-08-03 12:10 +0200
    [PATCH V3 net-next 19/21] net-next/hinic: Add Tx operation Aviad Krawczyk <aviad.krawczyk@huawei.com> - 2017-08-03 12:10 +0200
    [PATCH V3 net-next 21/21] net-next/hinic: Add select_queue and netpoll Aviad Krawczyk <aviad.krawczyk@huawei.com> - 2017-08-03 12:10 +0200
    [PATCH V3 net-next 06/21] net-next/hinic: Add api cmd commands Aviad Krawczyk <aviad.krawczyk@huawei.com> - 2017-08-03 12:20 +0200
    [PATCH V3 net-next 05/21] net-next/hinic: Add management messages Aviad Krawczyk <aviad.krawczyk@huawei.com> - 2017-08-03 12:20 +0200
    [PATCH V3 net-next 03/21] net-next/hinic: Initialize api cmd resources Aviad Krawczyk <aviad.krawczyk@huawei.com> - 2017-08-03 12:20 +0200
      Re: [PATCH V3 net-next 03/21] net-next/hinic: Initialize api cmd  resources David Miller <davem@davemloft.net> - 2017-08-04 00:40 +0200
        Re: [PATCH V3 net-next 03/21] net-next/hinic: Initialize api cmd  resources Aviad Krawczyk <aviad.krawczyk@huawei.com> - 2017-08-08 18:30 +0200
          Re: [PATCH V3 net-next 03/21] net-next/hinic: Initialize api cmd  resources David Miller <davem@davemloft.net> - 2017-08-08 19:10 +0200

#1702870 — [PATCH V3 net-next 00/21] Huawei HiNIC Ethernet Driver

FromAviad Krawczyk <aviad.krawczyk@huawei.com>
Date2017-08-03 12:00 +0200
Subject[PATCH V3 net-next 00/21] Huawei HiNIC Ethernet Driver
Message-ID<ual57-6Bs-3@gated-at.bofh.it>
The patch-set contains the support of the HiNIC Ethernet driver for
hinic family of PCIE Network Interface Cards.

The Huawei's PCIE HiNIC card is a new Ethernet card and hence there was
a need of a new driver.

The current driver is meant to be used for the Physical Function and there
would soon be a support for Virtual Function and more features once the
basic PF driver has been accepted.

Changes V2 -> V3:
1. replace dev_ functions by netif_ functions - Joe Perches comment
	https://lkml.org/lkml/2017/7/19/424
2. Fix the driver directory in MAINTAINERS file - Sergei Shtylyov comment
	https://lkml.org/lkml/2017/7/19/615
3. Add newline at the end of Makefile - David Miller comment
	https://lkml.org/lkml/2017/7/19/1345
4. Return a pointer as a val instead of in arg - Francois Romieu comment
	https://lkml.org/lkml/2017/7/19/1319
5. Change the error labels to err_xyz - Francois Romieu comment
	https://lkml.org/lkml/2017/7/19/1319
6. Remove check of Func Type in free function - Francois Romieu comment
	https://lkml.org/lkml/2017/7/19/1319
7. Remove !netdev check in remove function - Francois Romieu comment
	https://lkml.org/lkml/2017/7/19/1319
8. Use module_pci_driver - Francois Romieu comment
	https://lkml.org/lkml/2017/7/19/1319
9. Move the PCI device ID to the .c file - Francois Romieu comment
	https://lkml.org/lkml/2017/7/19/1319
10. Remove void * to avoid passing wrong ptr - Francois Romieu comment
	https://lkml.org/lkml/2017/7/19/1319

Changes V1 -> V2:
1. remove driver version - Andrew Lunn comment
	https://lkml.org/lkml/2017/7/12/372
2. replace kzalloc by devm_kzalloc for short clean - Andrew Lunn comment
	https://lkml.org/lkml/2017/7/12/372
3. replace pr_ functions by dev_ functions - Andrew Lunn comment
	https://lkml.org/lkml/2017/7/12/375
4. seperate last patch by moving ops to a new patch - Andrew Lunn comment
	https://lkml.org/lkml/2017/7/12/377

Aviad Krawczyk (21):
  net-next/hinic: Initialize hw interface
  net-next/hinic: Initialize hw device components
  net-next/hinic: Initialize api cmd resources
  net-next/hinic: Initialize api cmd hw
  net-next/hinic: Add management messages
  net-next/hinic: Add api cmd commands
  net-next/hinic: Add aeqs
  net-next/hinic: Add port management commands
  net-next/hinic: Add Rx mode and link event handler
  net-next/hinic: Add logical Txq and Rxq
  net-next/hinic: Add wq
  net-next/hinic: Add qp resources
  net-next/hinic: Set qp context
  net-next/hinic: Initialize cmdq
  net-next/hinic: Add ceqs
  net-next/hinic: Add cmdq commands
  net-next/hinic: Add cmdq completion handler
  net-next/hinic: Add Rx handler
  net-next/hinic: Add Tx operation
  net-next/hinic: Add ethtool and stats
  net-next/hinic: Add select_queue and netpoll

 Documentation/networking/hinic.txt                 |  125 +++
 MAINTAINERS                                        |    7 +
 drivers/net/ethernet/Kconfig                       |    1 +
 drivers/net/ethernet/Makefile                      |    1 +
 drivers/net/ethernet/huawei/Kconfig                |   19 +
 drivers/net/ethernet/huawei/Makefile               |    5 +
 drivers/net/ethernet/huawei/hinic/Kconfig          |   13 +
 drivers/net/ethernet/huawei/hinic/Makefile         |    6 +
 drivers/net/ethernet/huawei/hinic/hinic_common.c   |   80 ++
 drivers/net/ethernet/huawei/hinic/hinic_common.h   |   38 +
 drivers/net/ethernet/huawei/hinic/hinic_dev.h      |   64 ++
 .../net/ethernet/huawei/hinic/hinic_hw_api_cmd.c   |  975 +++++++++++++++++
 .../net/ethernet/huawei/hinic/hinic_hw_api_cmd.h   |  208 ++++
 drivers/net/ethernet/huawei/hinic/hinic_hw_cmdq.c  |  949 ++++++++++++++++
 drivers/net/ethernet/huawei/hinic/hinic_hw_cmdq.h  |  187 ++++
 drivers/net/ethernet/huawei/hinic/hinic_hw_csr.h   |  149 +++
 drivers/net/ethernet/huawei/hinic/hinic_hw_dev.c   | 1052 ++++++++++++++++++
 drivers/net/ethernet/huawei/hinic/hinic_hw_dev.h   |  239 ++++
 drivers/net/ethernet/huawei/hinic/hinic_hw_eqs.c   |  877 +++++++++++++++
 drivers/net/ethernet/huawei/hinic/hinic_hw_eqs.h   |  265 +++++
 drivers/net/ethernet/huawei/hinic/hinic_hw_if.c    |  353 ++++++
 drivers/net/ethernet/huawei/hinic/hinic_hw_if.h    |  272 +++++
 drivers/net/ethernet/huawei/hinic/hinic_hw_io.c    |  542 ++++++++++
 drivers/net/ethernet/huawei/hinic/hinic_hw_io.h    |   97 ++
 drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.c  |  598 ++++++++++
 drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.h  |  153 +++
 drivers/net/ethernet/huawei/hinic/hinic_hw_qp.c    |  874 +++++++++++++++
 drivers/net/ethernet/huawei/hinic/hinic_hw_qp.h    |  179 +++
 .../net/ethernet/huawei/hinic/hinic_hw_qp_ctxt.h   |  214 ++++
 drivers/net/ethernet/huawei/hinic/hinic_hw_wq.c    |  886 +++++++++++++++
 drivers/net/ethernet/huawei/hinic/hinic_hw_wq.h    |  117 ++
 drivers/net/ethernet/huawei/hinic/hinic_hw_wqe.h   |  368 +++++++
 drivers/net/ethernet/huawei/hinic/hinic_main.c     | 1139 ++++++++++++++++++++
 drivers/net/ethernet/huawei/hinic/hinic_port.c     |  403 +++++++
 drivers/net/ethernet/huawei/hinic/hinic_port.h     |  198 ++++
 drivers/net/ethernet/huawei/hinic/hinic_rx.c       |  520 +++++++++
 drivers/net/ethernet/huawei/hinic/hinic_rx.h       |   55 +
 drivers/net/ethernet/huawei/hinic/hinic_tx.c       |  516 +++++++++
 drivers/net/ethernet/huawei/hinic/hinic_tx.h       |   62 ++
 39 files changed, 12806 insertions(+)
 create mode 100644 Documentation/networking/hinic.txt
 create mode 100644 drivers/net/ethernet/huawei/Kconfig
 create mode 100644 drivers/net/ethernet/huawei/Makefile
 create mode 100644 drivers/net/ethernet/huawei/hinic/Kconfig
 create mode 100644 drivers/net/ethernet/huawei/hinic/Makefile
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_common.c
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_common.h
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_dev.h
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.c
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.h
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_hw_cmdq.c
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_hw_cmdq.h
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_hw_csr.h
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_hw_dev.c
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_hw_dev.h
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_hw_eqs.c
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_hw_eqs.h
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_hw_if.c
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_hw_if.h
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_hw_io.c
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_hw_io.h
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.c
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.h
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_hw_qp.c
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_hw_qp.h
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_hw_qp_ctxt.h
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_hw_wq.c
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_hw_wq.h
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_hw_wqe.h
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_main.c
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_port.c
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_port.h
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_rx.c
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_rx.h
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_tx.c
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_tx.h

-- 
1.9.1

[toc] | [next] | [standalone]


#1702871 — [PATCH V3 net-next 15/21] net-next/hinic: Add ceqs

FromAviad Krawczyk <aviad.krawczyk@huawei.com>
Date2017-08-03 12:00 +0200
Subject[PATCH V3 net-next 15/21] net-next/hinic: Add ceqs
Message-ID<ual58-6Bs-29@gated-at.bofh.it>
In reply to#1702870
Initialize the completion event queues and handle ceq events by calling
the registered handlers. Used for cmdq command completion.

Signed-off-by: Aviad Krawczyk <aviad.krawczyk@huawei.com>
Signed-off-by: Zhao Chen <zhaochen6@huawei.com>
---
 drivers/net/ethernet/huawei/hinic/hinic_hw_cmdq.c |  16 ++
 drivers/net/ethernet/huawei/hinic/hinic_hw_csr.h  |  29 +++
 drivers/net/ethernet/huawei/hinic/hinic_hw_dev.c  |   7 +-
 drivers/net/ethernet/huawei/hinic/hinic_hw_eqs.c  | 290 +++++++++++++++++++++-
 drivers/net/ethernet/huawei/hinic/hinic_hw_eqs.h  |  75 ++++++
 drivers/net/ethernet/huawei/hinic/hinic_hw_io.c   |  15 +-
 drivers/net/ethernet/huawei/hinic/hinic_hw_io.h   |   3 +
 7 files changed, 427 insertions(+), 8 deletions(-)

diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_cmdq.c b/drivers/net/ethernet/huawei/hinic/hinic_hw_cmdq.c
index ef8b715..f734ca2 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_cmdq.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_cmdq.c
@@ -27,6 +27,7 @@
 #include <asm/byteorder.h>
 
 #include "hinic_hw_if.h"
+#include "hinic_hw_eqs.h"
 #include "hinic_hw_mgmt.h"
 #include "hinic_hw_wq.h"
 #include "hinic_hw_cmdq.h"
@@ -110,6 +111,16 @@ int hinic_cmdq_direct_resp(struct hinic_cmdqs *cmdqs,
 }
 
 /**
+ * cmdq_ceq_handler - cmdq completion event handler
+ * @handle: private data for the handler(cmdqs)
+ * @ceqe_data: ceq element data
+ **/
+static void cmdq_ceq_handler(void *handle, u32 ceqe_data)
+{
+	/* should be implemented */
+}
+
+/**
  * cmdq_init_queue_ctxt - init the queue ctxt of a cmdq
  * @cmdq_ctxt: cmdq ctxt to initialize
  * @cmdq: the cmdq
@@ -322,6 +333,8 @@ int hinic_init_cmdqs(struct hinic_cmdqs *cmdqs, struct hinic_hwif *hwif,
 		goto err_cmdq_ctxt;
 	}
 
+	hinic_ceq_register_cb(&func_to_io->ceqs, HINIC_CEQ_CMDQ, cmdqs,
+			      cmdq_ceq_handler);
 	return 0;
 
 err_cmdq_ctxt:
@@ -342,10 +355,13 @@ int hinic_init_cmdqs(struct hinic_cmdqs *cmdqs, struct hinic_hwif *hwif,
  **/
 void hinic_free_cmdqs(struct hinic_cmdqs *cmdqs)
 {
+	struct hinic_func_to_io *func_to_io = cmdqs_to_func_to_io(cmdqs);
 	struct hinic_hwif *hwif = cmdqs->hwif;
 	struct pci_dev *pdev = hwif->pdev;
 	enum hinic_cmdq_type cmdq_type;
 
+	hinic_ceq_unregister_cb(&func_to_io->ceqs, HINIC_CEQ_CMDQ);
+
 	cmdq_type = HINIC_CMDQ_SYNC;
 	for (; cmdq_type < HINIC_MAX_CMDQ_TYPES; cmdq_type++)
 		free_cmdq(&cmdqs->cmdq[cmdq_type]);
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_csr.h b/drivers/net/ethernet/huawei/hinic/hinic_hw_csr.h
index 1f57301..10b8c7b 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_csr.h
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_csr.h
@@ -81,27 +81,44 @@
 
 /* EQ registers */
 #define HINIC_AEQ_MTT_OFF_BASE_ADDR                     0x200
+#define HINIC_CEQ_MTT_OFF_BASE_ADDR                     0x400
 
 #define HINIC_EQ_MTT_OFF_STRIDE                         0x40
 
 #define HINIC_CSR_AEQ_MTT_OFF(id)                       \
 	(HINIC_AEQ_MTT_OFF_BASE_ADDR + (id) * HINIC_EQ_MTT_OFF_STRIDE)
 
+#define HINIC_CSR_CEQ_MTT_OFF(id)                       \
+	(HINIC_CEQ_MTT_OFF_BASE_ADDR + (id) * HINIC_EQ_MTT_OFF_STRIDE)
+
 #define HINIC_CSR_EQ_PAGE_OFF_STRIDE                    8
 
 #define HINIC_CSR_AEQ_HI_PHYS_ADDR_REG(q_id, pg_num)    \
 	(HINIC_CSR_AEQ_MTT_OFF(q_id) + \
 	 (pg_num) * HINIC_CSR_EQ_PAGE_OFF_STRIDE)
 
+#define HINIC_CSR_CEQ_HI_PHYS_ADDR_REG(q_id, pg_num)    \
+	(HINIC_CSR_CEQ_MTT_OFF(q_id) +          \
+	 (pg_num) * HINIC_CSR_EQ_PAGE_OFF_STRIDE)
+
 #define HINIC_CSR_AEQ_LO_PHYS_ADDR_REG(q_id, pg_num)    \
 	(HINIC_CSR_AEQ_MTT_OFF(q_id) + \
 	 (pg_num) * HINIC_CSR_EQ_PAGE_OFF_STRIDE + 4)
 
+#define HINIC_CSR_CEQ_LO_PHYS_ADDR_REG(q_id, pg_num)    \
+	(HINIC_CSR_CEQ_MTT_OFF(q_id) +  \
+	 (pg_num) * HINIC_CSR_EQ_PAGE_OFF_STRIDE + 4)
+
 #define HINIC_AEQ_CTRL_0_ADDR_BASE                      0xE00
 #define HINIC_AEQ_CTRL_1_ADDR_BASE                      0xE04
 #define HINIC_AEQ_CONS_IDX_ADDR_BASE                    0xE08
 #define HINIC_AEQ_PROD_IDX_ADDR_BASE                    0xE0C
 
+#define HINIC_CEQ_CTRL_0_ADDR_BASE                      0x1000
+#define HINIC_CEQ_CTRL_1_ADDR_BASE                      0x1004
+#define HINIC_CEQ_CONS_IDX_ADDR_BASE                    0x1008
+#define HINIC_CEQ_PROD_IDX_ADDR_BASE                    0x100C
+
 #define HINIC_EQ_OFF_STRIDE                             0x80
 
 #define HINIC_CSR_AEQ_CTRL_0_ADDR(idx)                  \
@@ -116,4 +133,16 @@
 #define HINIC_CSR_AEQ_PROD_IDX_ADDR(idx)                \
 	(HINIC_AEQ_PROD_IDX_ADDR_BASE + (idx) * HINIC_EQ_OFF_STRIDE)
 
+#define HINIC_CSR_CEQ_CTRL_0_ADDR(idx)                  \
+	(HINIC_CEQ_CTRL_0_ADDR_BASE + (idx) * HINIC_EQ_OFF_STRIDE)
+
+#define HINIC_CSR_CEQ_CTRL_1_ADDR(idx)                  \
+	(HINIC_CEQ_CTRL_1_ADDR_BASE + (idx) * HINIC_EQ_OFF_STRIDE)
+
+#define HINIC_CSR_CEQ_CONS_IDX_ADDR(idx)                \
+	(HINIC_CEQ_CONS_IDX_ADDR_BASE + (idx) * HINIC_EQ_OFF_STRIDE)
+
+#define HINIC_CSR_CEQ_PROD_IDX_ADDR(idx)                \
+	(HINIC_CEQ_PROD_IDX_ADDR_BASE + (idx) * HINIC_EQ_OFF_STRIDE)
+
 #endif
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_dev.c b/drivers/net/ethernet/huawei/hinic/hinic_hw_dev.c
index e43060a..77835c4 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_dev.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_dev.c
@@ -282,6 +282,7 @@ int hinic_hwdev_ifup(struct hinic_hwdev *hwdev)
 	struct pci_dev *pdev = hwif->pdev;
 	int num_qps = nic_cap->num_qps;
 	int max_qps = nic_cap->max_qps;
+	struct msix_entry *ceq_msix_entries;
 	struct msix_entry *sq_msix_entries;
 	struct msix_entry *rq_msix_entries;
 	u16 base_qpn;
@@ -295,7 +296,11 @@ int hinic_hwdev_ifup(struct hinic_hwdev *hwdev)
 
 	num_aeqs = HINIC_HWIF_NUM_AEQS(hwif);
 	num_ceqs = HINIC_HWIF_NUM_CEQS(hwif);
-	err = hinic_io_init(func_to_io, hwif, max_qps, 0, NULL);
+
+	ceq_msix_entries = &hwdev->msix_entries[num_aeqs];
+
+	err = hinic_io_init(func_to_io, hwif, max_qps, num_ceqs,
+			    ceq_msix_entries);
 	if (err) {
 		dev_err(&pdev->dev, "Failed to init IO channel\n");
 		return err;
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_eqs.c b/drivers/net/ethernet/huawei/hinic/hinic_hw_eqs.c
index 731b0c1..b15fb8e 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_eqs.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_eqs.c
@@ -37,14 +37,21 @@
 
 #define GET_EQ_NUM_ELEMS_IN_PG(eq, pg_size)     ((pg_size) / (eq)->elem_size)
 
-#define EQ_CONS_IDX_REG_ADDR(eq)        HINIC_CSR_AEQ_CONS_IDX_ADDR((eq)->q_id)
-#define EQ_PROD_IDX_REG_ADDR(eq)        HINIC_CSR_AEQ_PROD_IDX_ADDR((eq)->q_id)
+#define EQ_CONS_IDX_REG_ADDR(eq)        (((eq)->type == HINIC_AEQ) ? \
+			HINIC_CSR_AEQ_CONS_IDX_ADDR((eq)->q_id) : \
+			HINIC_CSR_CEQ_CONS_IDX_ADDR((eq)->q_id))
 
-#define EQ_HI_PHYS_ADDR_REG(eq, pg_num) \
-		HINIC_CSR_AEQ_HI_PHYS_ADDR_REG((eq)->q_id, pg_num)
+#define EQ_PROD_IDX_REG_ADDR(eq)        (((eq)->type == HINIC_AEQ) ? \
+			HINIC_CSR_AEQ_PROD_IDX_ADDR((eq)->q_id) : \
+			HINIC_CSR_CEQ_PROD_IDX_ADDR((eq)->q_id))
 
-#define EQ_LO_PHYS_ADDR_REG(eq, pg_num) \
-		HINIC_CSR_AEQ_LO_PHYS_ADDR_REG((eq)->q_id, pg_num)
+#define EQ_HI_PHYS_ADDR_REG(eq, pg_num) (((eq)->type == HINIC_AEQ) ? \
+			HINIC_CSR_AEQ_HI_PHYS_ADDR_REG((eq)->q_id, pg_num) : \
+			HINIC_CSR_CEQ_HI_PHYS_ADDR_REG((eq)->q_id, pg_num))
+
+#define EQ_LO_PHYS_ADDR_REG(eq, pg_num) (((eq)->type == HINIC_AEQ) ? \
+			HINIC_CSR_AEQ_LO_PHYS_ADDR_REG((eq)->q_id, pg_num) : \
+			HINIC_CSR_CEQ_LO_PHYS_ADDR_REG((eq)->q_id, pg_num))
 
 #define GET_EQ_ELEMENT(eq, idx)         \
 		((eq)->virt_addr[(idx) / (eq)->num_elem_in_pg] + \
@@ -53,8 +60,13 @@
 #define GET_AEQ_ELEM(eq, idx)           ((struct hinic_aeq_elem *) \
 					GET_EQ_ELEMENT(eq, idx))
 
+#define GET_CEQ_ELEM(eq, idx)           ((u32 *) \
+					 GET_EQ_ELEMENT(eq, idx))
+
 #define GET_CURR_AEQ_ELEM(eq)           GET_AEQ_ELEM(eq, (eq)->cons_idx)
 
+#define GET_CURR_CEQ_ELEM(eq)           GET_CEQ_ELEM(eq, (eq)->cons_idx)
+
 #define PAGE_IN_4K(page_size)           ((page_size) >> 12)
 #define EQ_SET_HW_PAGE_SIZE_VAL(eq)     (ilog2(PAGE_IN_4K((eq)->page_size)))
 
@@ -63,13 +75,29 @@
 
 #define EQ_MAX_PAGES                    8
 
+#define CEQE_TYPE_SHIFT                 23
+#define CEQE_TYPE_MASK                  0x7
+
+#define CEQE_TYPE(ceqe)                 (((ceqe) >> CEQE_TYPE_SHIFT) &  \
+					 CEQE_TYPE_MASK)
+
+#define CEQE_DATA_MASK                  0x3FFFFFF
+#define CEQE_DATA(ceqe)                 ((ceqe) & CEQE_DATA_MASK)
+
 #define aeq_to_aeqs(eq)                 \
 		container_of((eq) - (eq)->q_id, struct hinic_aeqs, aeq[0])
 
+#define ceq_to_ceqs(eq)                 \
+		container_of((eq) - (eq)->q_id, struct hinic_ceqs, ceq[0])
+
 #define work_to_aeq_work(work)          \
 		container_of(work, struct hinic_eq_work, work)
 
 #define DMA_ATTR_AEQ_DEFAULT            0
+#define DMA_ATTR_CEQ_DEFAULT            0
+
+/* No coalescence */
+#define THRESH_CEQ_DEFAULT              0
 
 enum eq_int_mode {
 	EQ_INT_MODE_ARMED,
@@ -118,6 +146,42 @@ void hinic_aeq_unregister_hw_cb(struct hinic_aeqs *aeqs,
 	hwe_cb->hwe_handler = NULL;
 }
 
+/**
+ * hinic_ceq_register_cb - register CEQ callback for specific event
+ * @ceqs: pointer to Completion eqs part of the chip
+ * @event: ceq event to register callback for it
+ * @handle: private data will be used by the callback
+ * @handler: callback function
+ **/
+void hinic_ceq_register_cb(struct hinic_ceqs *ceqs,
+			   enum hinic_ceq_type event, void *handle,
+			   void (*handler)(void *handle, u32 ceqe_data))
+{
+	struct hinic_ceq_cb *ceq_cb = &ceqs->ceq_cb[event];
+
+	ceq_cb->handler = handler;
+	ceq_cb->handle = handle;
+	ceq_cb->ceqe_state = HINIC_EQE_ENABLED;
+}
+
+/**
+ * hinic_ceq_unregister_cb - unregister the CEQ callback for specific event
+ * @ceqs: pointer to Completion eqs part of the chip
+ * @event: ceq event to unregister callback for it
+ **/
+void hinic_ceq_unregister_cb(struct hinic_ceqs *ceqs,
+			     enum hinic_ceq_type event)
+{
+	struct hinic_ceq_cb *ceq_cb = &ceqs->ceq_cb[event];
+
+	ceq_cb->ceqe_state &= ~HINIC_EQE_ENABLED;
+
+	while (ceq_cb->ceqe_state & HINIC_EQE_RUNNING)
+		schedule();
+
+	ceq_cb->handler = NULL;
+}
+
 static u8 eq_cons_idx_checksum_set(u32 val)
 {
 	u8 checksum = 0;
@@ -220,6 +284,70 @@ static void aeq_irq_handler(struct hinic_eq *eq)
 }
 
 /**
+ * ceq_event_handler - handler for the ceq events
+ * @ceqs: ceqs part of the chip
+ * @ceqe: ceq element that describes the event
+ **/
+static void ceq_event_handler(struct hinic_ceqs *ceqs, u32 ceqe)
+{
+	struct hinic_hwif *hwif = ceqs->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+	enum hinic_ceq_type event = CEQE_TYPE(ceqe);
+	u32 ceqe_data = CEQE_DATA(ceqe);
+	struct hinic_ceq_cb *ceq_cb;
+	unsigned long eqe_state;
+
+	if (event >= HINIC_MAX_CEQ_EVENTS) {
+		dev_err(&pdev->dev, "Unknown CEQ event, event = %d\n", event);
+		return;
+	}
+
+	ceq_cb = &ceqs->ceq_cb[event];
+
+	eqe_state = cmpxchg(&ceq_cb->ceqe_state,
+			    HINIC_EQE_ENABLED,
+			    HINIC_EQE_ENABLED | HINIC_EQE_RUNNING);
+
+	if ((eqe_state == HINIC_EQE_ENABLED) && (ceq_cb->handler))
+		ceq_cb->handler(ceq_cb->handle, ceqe_data);
+	else
+		dev_err(&pdev->dev, "Unhandled CEQ Event %d\n", event);
+
+	ceq_cb->ceqe_state &= ~HINIC_EQE_RUNNING;
+}
+
+/**
+ * ceq_irq_handler - handler for the CEQ event
+ * @eq: the Completion Event Queue that received the event
+ **/
+static void ceq_irq_handler(struct hinic_eq *eq)
+{
+	struct hinic_ceqs *ceqs = ceq_to_ceqs(eq);
+	u32 ceqe;
+	int i;
+
+	for (i = 0; i < eq->q_len; i++) {
+		ceqe = *(GET_CURR_CEQ_ELEM(eq));
+
+		/* Data in HW is in Big endian Format */
+		ceqe = be32_to_cpu(ceqe);
+
+		/* HW toggles the wrapped bit, when it adds eq element event */
+		if (HINIC_EQ_ELEM_DESC_GET(ceqe, WRAPPED) == eq->wrapped)
+			break;
+
+		ceq_event_handler(ceqs, ceqe);
+
+		eq->cons_idx++;
+
+		if (eq->cons_idx == eq->q_len) {
+			eq->cons_idx = 0;
+			eq->wrapped = !eq->wrapped;
+		}
+	}
+}
+
+/**
  * eq_irq_handler - handler for the EQ event
  * @data: the Event Queue that received the event
  **/
@@ -229,6 +357,8 @@ static void eq_irq_handler(void *data)
 
 	if (eq->type == HINIC_AEQ)
 		aeq_irq_handler(eq);
+	else if (eq->type == HINIC_CEQ)
+		ceq_irq_handler(eq);
 
 	eq_update_ci(eq);
 }
@@ -246,6 +376,17 @@ static void eq_irq_work(struct work_struct *work)
 }
 
 /**
+ * ceq_tasklet - the tasklet of the EQ that received the event
+ * @ceq_data: the eq
+ **/
+static void ceq_tasklet(unsigned long ceq_data)
+{
+	struct hinic_eq *ceq = (struct hinic_eq *)ceq_data;
+
+	eq_irq_handler(ceq);
+}
+
+/**
  * aeq_interrupt - aeq interrupt handler
  * @irq: irq number
  * @data: the Async Event Queue that collected the event
@@ -270,6 +411,24 @@ static irqreturn_t aeq_interrupt(int irq, void *data)
 }
 
 /**
+ * ceq_interrupt - ceq interrupt handler
+ * @irq: irq number
+ * @data: the Completion Event Queue that collected the event
+ **/
+static irqreturn_t ceq_interrupt(int irq, void *data)
+{
+	struct hinic_eq *ceq = (struct hinic_eq *)data;
+	struct hinic_hwif *hwif = ceq->hwif;
+
+	/* clear resend timer cnt register */
+	hinic_msix_attr_cnt_clear(hwif, ceq->msix_entry.entry);
+
+	tasklet_schedule(&ceq->ceq_tasklet);
+
+	return IRQ_HANDLED;
+}
+
+/**
  * set_eq_ctrls - setting eq's ctrl registers
  * @eq: the Event Queue for setting
  **/
@@ -320,6 +479,44 @@ static void set_eq_ctrls(struct hinic_eq *eq)
 		val |= ctrl1;
 
 		hinic_hwif_write_reg(hwif, addr, val);
+	} else if (type == HINIC_CEQ) {
+		/* RMW Ctrl0 */
+		addr = HINIC_CSR_CEQ_CTRL_0_ADDR(eq->q_id);
+
+		val = hinic_hwif_read_reg(hwif, addr);
+
+		val = HINIC_CEQ_CTRL_0_CLEAR(val, INTR_IDX)     &
+		      HINIC_CEQ_CTRL_0_CLEAR(val, DMA_ATTR)     &
+		      HINIC_CEQ_CTRL_0_CLEAR(val, KICK_THRESH)  &
+		      HINIC_CEQ_CTRL_0_CLEAR(val, PCI_INTF_IDX) &
+		      HINIC_CEQ_CTRL_0_CLEAR(val, INTR_MODE);
+
+		ctrl0 = HINIC_CEQ_CTRL_0_SET(msix_entry->entry, INTR_IDX)    |
+			HINIC_CEQ_CTRL_0_SET(DMA_ATTR_CEQ_DEFAULT, DMA_ATTR) |
+			HINIC_CEQ_CTRL_0_SET(THRESH_CEQ_DEFAULT, KICK_THRESH) |
+			HINIC_CEQ_CTRL_0_SET(pci_intf_idx, PCI_INTF_IDX) |
+			HINIC_CEQ_CTRL_0_SET(EQ_INT_MODE_ARMED, INTR_MODE);
+
+		val |= ctrl0;
+
+		hinic_hwif_write_reg(hwif, addr, val);
+
+		/* RMW Ctrl1 */
+		addr = HINIC_CSR_CEQ_CTRL_1_ADDR(eq->q_id);
+
+		page_size_val = EQ_SET_HW_PAGE_SIZE_VAL(eq);
+
+		val = hinic_hwif_read_reg(hwif, addr);
+
+		val = HINIC_CEQ_CTRL_1_CLEAR(val, LEN) &
+		      HINIC_CEQ_CTRL_1_CLEAR(val, PAGE_SIZE);
+
+		ctrl1 = HINIC_CEQ_CTRL_1_SET(eq->q_len, LEN) |
+			HINIC_CEQ_CTRL_1_SET(page_size_val, PAGE_SIZE);
+
+		val |= ctrl1;
+
+		hinic_hwif_write_reg(hwif, addr, val);
 	}
 }
 
@@ -342,6 +539,24 @@ static void aeq_elements_init(struct hinic_eq *eq, u32 init_val)
 }
 
 /**
+ * ceq_elements_init - Initialize all the elements in the ceq
+ * @eq: the event queue
+ * @init_val: value to init with it the elements
+ **/
+static void ceq_elements_init(struct hinic_eq *eq, u32 init_val)
+{
+	int i;
+	u32 *ceqe;
+
+	for (i = 0; i < eq->q_len; i++) {
+		ceqe = GET_CEQ_ELEM(eq, i);
+		*(ceqe) = cpu_to_be32(init_val);
+	}
+
+	wmb();  /* Write the initilzation values */
+}
+
+/**
  * alloc_eq_pages - allocate the pages for the queue
  * @eq: the event queue
  *
@@ -393,6 +608,8 @@ static int alloc_eq_pages(struct hinic_eq *eq)
 
 	if (eq->type == HINIC_AEQ)
 		aeq_elements_init(eq, init_val);
+	else if (eq->type == HINIC_CEQ)
+		ceq_elements_init(eq, init_val);
 
 	return 0;
 
@@ -462,6 +679,8 @@ static int init_eq(struct hinic_eq *eq, struct hinic_hwif *hwif,
 
 	if (type == HINIC_AEQ) {
 		eq->elem_size = HINIC_AEQE_SIZE;
+	} else if (type == HINIC_CEQ) {
+		eq->elem_size = HINIC_CEQE_SIZE;
 	} else {
 		dev_err(&pdev->dev, "Invalid EQ type\n");
 		return -EINVAL;
@@ -495,6 +714,9 @@ static int init_eq(struct hinic_eq *eq, struct hinic_hwif *hwif,
 		struct hinic_eq_work *aeq_work = &eq->aeq_work;
 
 		INIT_WORK(&aeq_work->work, eq_irq_work);
+	} else if (type == HINIC_CEQ) {
+		tasklet_init(&eq->ceq_tasklet, ceq_tasklet,
+			     (unsigned long)eq);
 	}
 
 	/* set the attributes of the msix entry */
@@ -508,6 +730,9 @@ static int init_eq(struct hinic_eq *eq, struct hinic_hwif *hwif,
 	if (type == HINIC_AEQ)
 		err = request_irq(entry.vector, aeq_interrupt, 0,
 				  "hinic_aeq", eq);
+	else if (type == HINIC_CEQ)
+		err = request_irq(entry.vector, ceq_interrupt, 0,
+				  "hinic_ceq", eq);
 
 	if (err) {
 		dev_err(&pdev->dev, "Failed to request irq for the EQ\n");
@@ -535,6 +760,8 @@ static void remove_eq(struct hinic_eq *eq)
 		struct hinic_eq_work *aeq_work = &eq->aeq_work;
 
 		cancel_work_sync(&aeq_work->work);
+	} else if (eq->type == HINIC_CEQ) {
+		tasklet_kill(&eq->ceq_tasklet);
 	}
 
 	free_eq_pages(eq);
@@ -597,3 +824,54 @@ void hinic_aeqs_free(struct hinic_aeqs *aeqs)
 
 	destroy_workqueue(aeqs->workq);
 }
+
+/**
+ * hinic_ceqs_init - init all the ceqs
+ * @ceqs: ceqs part of the chip
+ * @hwif: the hardware interface of a pci function device
+ * @num_ceqs: number of CEQs
+ * @q_len: number of EQ elements
+ * @page_size: the page size of the event queue
+ * @msix_entries: msix entries associated with the event queues
+ *
+ * Return 0 - Success, Negative - Failure
+ **/
+int hinic_ceqs_init(struct hinic_ceqs *ceqs, struct hinic_hwif *hwif,
+		    int num_ceqs, u32 q_len, u32 page_size,
+		    struct msix_entry *msix_entries)
+{
+	struct pci_dev *pdev = hwif->pdev;
+	int i, q_id, err;
+
+	ceqs->hwif = hwif;
+	ceqs->num_ceqs = num_ceqs;
+
+	for (q_id = 0; q_id < num_ceqs; q_id++) {
+		err = init_eq(&ceqs->ceq[q_id], hwif, HINIC_CEQ, q_id, q_len,
+			      page_size, msix_entries[q_id]);
+		if (err) {
+			dev_err(&pdev->dev, "Failed to init ceq %d\n", q_id);
+			goto err_init_ceq;
+		}
+	}
+
+	return 0;
+
+err_init_ceq:
+	for (i = 0; i < q_id; i++)
+		remove_eq(&ceqs->ceq[i]);
+
+	return err;
+}
+
+/**
+ * hinic_ceqs_free - free all the ceqs
+ * @ceqs: ceqs part of the chip
+ **/
+void hinic_ceqs_free(struct hinic_ceqs *ceqs)
+{
+	int q_id;
+
+	for (q_id = 0; q_id < ceqs->num_ceqs; q_id++)
+		remove_eq(&ceqs->ceq[q_id]);
+}
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_eqs.h b/drivers/net/ethernet/huawei/hinic/hinic_hw_eqs.h
index ca584c0..ecb9c2b 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_eqs.h
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_eqs.h
@@ -21,6 +21,7 @@
 #include <linux/pci.h>
 #include <linux/sizes.h>
 #include <linux/bitops.h>
+#include <linux/interrupt.h>
 
 #include "hinic_hw_if.h"
 
@@ -58,6 +59,40 @@
 			((val) & (~(HINIC_AEQ_CTRL_1_##member##_MASK \
 			 << HINIC_AEQ_CTRL_1_##member##_SHIFT)))
 
+#define HINIC_CEQ_CTRL_0_INTR_IDX_SHIFT         0
+#define HINIC_CEQ_CTRL_0_DMA_ATTR_SHIFT         12
+#define HINIC_CEQ_CTRL_0_KICK_THRESH_SHIFT      20
+#define HINIC_CEQ_CTRL_0_PCI_INTF_IDX_SHIFT     24
+#define HINIC_CEQ_CTRL_0_INTR_MODE_SHIFT        31
+
+#define HINIC_CEQ_CTRL_0_INTR_IDX_MASK          0x3FF
+#define HINIC_CEQ_CTRL_0_DMA_ATTR_MASK          0x3F
+#define HINIC_CEQ_CTRL_0_KICK_THRESH_MASK       0xF
+#define HINIC_CEQ_CTRL_0_PCI_INTF_IDX_MASK      0x3
+#define HINIC_CEQ_CTRL_0_INTR_MODE_MASK         0x1
+
+#define HINIC_CEQ_CTRL_0_SET(val, member)       \
+			(((u32)(val) & HINIC_CEQ_CTRL_0_##member##_MASK) << \
+			 HINIC_CEQ_CTRL_0_##member##_SHIFT)
+
+#define HINIC_CEQ_CTRL_0_CLEAR(val, member)     \
+			((val) & (~(HINIC_CEQ_CTRL_0_##member##_MASK \
+			 << HINIC_CEQ_CTRL_0_##member##_SHIFT)))
+
+#define HINIC_CEQ_CTRL_1_LEN_SHIFT              0
+#define HINIC_CEQ_CTRL_1_PAGE_SIZE_SHIFT        28
+
+#define HINIC_CEQ_CTRL_1_LEN_MASK               0x1FFFFF
+#define HINIC_CEQ_CTRL_1_PAGE_SIZE_MASK         0xF
+
+#define HINIC_CEQ_CTRL_1_SET(val, member)       \
+			(((u32)(val) & HINIC_CEQ_CTRL_1_##member##_MASK) << \
+			 HINIC_CEQ_CTRL_1_##member##_SHIFT)
+
+#define HINIC_CEQ_CTRL_1_CLEAR(val, member)     \
+			((val) & (~(HINIC_CEQ_CTRL_1_##member##_MASK \
+			 << HINIC_CEQ_CTRL_1_##member##_SHIFT)))
+
 #define HINIC_EQ_ELEM_DESC_TYPE_SHIFT           0
 #define HINIC_EQ_ELEM_DESC_SRC_SHIFT            7
 #define HINIC_EQ_ELEM_DESC_SIZE_SHIFT           8
@@ -95,14 +130,17 @@
 			 << HINIC_EQ_CI_##member##_SHIFT)))
 
 #define HINIC_MAX_AEQS                  4
+#define HINIC_MAX_CEQS                  32
 
 #define HINIC_AEQE_SIZE                 64
+#define HINIC_CEQE_SIZE                 4
 
 #define HINIC_AEQE_DESC_SIZE            4
 #define HINIC_AEQE_DATA_SIZE            \
 			(HINIC_AEQE_SIZE - HINIC_AEQE_DESC_SIZE)
 
 #define HINIC_DEFAULT_AEQ_LEN           64
+#define HINIC_DEFAULT_CEQ_LEN           1024
 
 #define HINIC_EQ_PAGE_SIZE              SZ_4K
 
@@ -110,6 +148,7 @@
 
 enum hinic_eq_type {
 	HINIC_AEQ,
+	HINIC_CEQ,
 };
 
 enum hinic_aeq_type {
@@ -118,6 +157,12 @@ enum hinic_aeq_type {
 	HINIC_MAX_AEQ_EVENTS,
 };
 
+enum hinic_ceq_type {
+	HINIC_CEQ_CMDQ = 3,
+
+	HINIC_MAX_CEQ_EVENTS,
+};
+
 enum hinic_eqe_state {
 	HINIC_EQE_ENABLED = BIT(0),
 	HINIC_EQE_RUNNING = BIT(1),
@@ -154,6 +199,8 @@ struct hinic_eq {
 	void                    **virt_addr;
 
 	struct hinic_eq_work    aeq_work;
+
+	struct tasklet_struct   ceq_tasklet;
 };
 
 struct hinic_hw_event_cb {
@@ -173,6 +220,21 @@ struct hinic_aeqs {
 	struct workqueue_struct *workq;
 };
 
+struct hinic_ceq_cb {
+	void    (*handler)(void *handle, u32 ceqe_data);
+	void                    *handle;
+	enum hinic_eqe_state    ceqe_state;
+};
+
+struct hinic_ceqs {
+	struct hinic_hwif       *hwif;
+
+	struct hinic_eq         ceq[HINIC_MAX_CEQS];
+	int                     num_ceqs;
+
+	struct hinic_ceq_cb     ceq_cb[HINIC_MAX_CEQ_EVENTS];
+};
+
 void hinic_aeq_register_hw_cb(struct hinic_aeqs *aeqs,
 			      enum hinic_aeq_type event, void *handle,
 			      void (*hwe_handler)(void *handle, void *data,
@@ -181,10 +243,23 @@ void hinic_aeq_register_hw_cb(struct hinic_aeqs *aeqs,
 void hinic_aeq_unregister_hw_cb(struct hinic_aeqs *aeqs,
 				enum hinic_aeq_type event);
 
+void hinic_ceq_register_cb(struct hinic_ceqs *ceqs,
+			   enum hinic_ceq_type event, void *handle,
+			   void (*ceq_cb)(void *handle, u32 ceqe_data));
+
+void hinic_ceq_unregister_cb(struct hinic_ceqs *ceqs,
+			     enum hinic_ceq_type event);
+
 int hinic_aeqs_init(struct hinic_aeqs *aeqs, struct hinic_hwif *hwif,
 		    int num_aeqs, u32 q_len, u32 page_size,
 		    struct msix_entry *msix_entries);
 
 void hinic_aeqs_free(struct hinic_aeqs *aeqs);
 
+int hinic_ceqs_init(struct hinic_ceqs *ceqs, struct hinic_hwif *hwif,
+		    int num_ceqs, u32 q_len, u32 page_size,
+		    struct msix_entry *msix_entries);
+
+void hinic_ceqs_free(struct hinic_ceqs *ceqs);
+
 #endif
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_io.c b/drivers/net/ethernet/huawei/hinic/hinic_hw_io.c
index 71f5d63..95dc57b 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_io.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_io.c
@@ -25,6 +25,7 @@
 #include <linux/err.h>
 
 #include "hinic_hw_if.h"
+#include "hinic_hw_eqs.h"
 #include "hinic_hw_wqe.h"
 #include "hinic_hw_wq.h"
 #include "hinic_hw_cmdq.h"
@@ -464,10 +465,18 @@ int hinic_io_init(struct hinic_func_to_io *func_to_io,
 	func_to_io->qps = NULL;
 	func_to_io->max_qps = max_qps;
 
+	err = hinic_ceqs_init(&func_to_io->ceqs, hwif, num_ceqs,
+			      HINIC_DEFAULT_CEQ_LEN, HINIC_EQ_PAGE_SIZE,
+			      ceq_msix_entries);
+	if (err) {
+		dev_err(&pdev->dev, "Failed to init CEQs\n");
+		return err;
+	}
+
 	err = hinic_wqs_alloc(&func_to_io->wqs, 2 * max_qps, hwif);
 	if (err) {
 		dev_err(&pdev->dev, "Failed to allocate WQS for IO\n");
-		return err;
+		goto err_wqs_alloc;
 	}
 
 	func_to_io->db_base = pci_ioremap_bar(pdev, HINIC_PCI_DB_BAR);
@@ -508,6 +517,9 @@ int hinic_io_init(struct hinic_func_to_io *func_to_io,
 
 err_db_ioremap:
 	hinic_wqs_free(&func_to_io->wqs);
+
+err_wqs_alloc:
+	hinic_ceqs_free(&func_to_io->ceqs);
 	return err;
 }
 
@@ -526,4 +538,5 @@ void hinic_io_free(struct hinic_func_to_io *func_to_io)
 
 	iounmap(func_to_io->db_base);
 	hinic_wqs_free(&func_to_io->wqs);
+	hinic_ceqs_free(&func_to_io->ceqs);
 }
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_io.h b/drivers/net/ethernet/huawei/hinic/hinic_hw_io.h
index 60d77b34..cfc21ba 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_io.h
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_io.h
@@ -22,6 +22,7 @@
 #include <linux/sizes.h>
 
 #include "hinic_hw_if.h"
+#include "hinic_hw_eqs.h"
 #include "hinic_hw_wq.h"
 #include "hinic_hw_cmdq.h"
 #include "hinic_hw_qp.h"
@@ -46,6 +47,8 @@ struct hinic_free_db_area {
 struct hinic_func_to_io {
 	struct hinic_hwif       *hwif;
 
+	struct hinic_ceqs       ceqs;
+
 	struct hinic_wqs        wqs;
 
 	struct hinic_wq         *sq_wq;
-- 
1.9.1

[toc] | [prev] | [next] | [standalone]


#1702873 — [PATCH V3 net-next 04/21] net-next/hinic: Initialize api cmd hw

FromAviad Krawczyk <aviad.krawczyk@huawei.com>
Date2017-08-03 12:00 +0200
Subject[PATCH V3 net-next 04/21] net-next/hinic: Initialize api cmd hw
Message-ID<ual59-6Bs-45@gated-at.bofh.it>
In reply to#1702870
Update the hardware about api cmd resources and initialize it.

Signed-off-by: Aviad Krawczyk <aviad.krawczyk@huawei.com>
Signed-off-by: Zhao Chen <zhaochen6@huawei.com>
---
 .../net/ethernet/huawei/hinic/hinic_hw_api_cmd.c   | 173 ++++++++++++++++++++-
 .../net/ethernet/huawei/hinic/hinic_hw_api_cmd.h   |  38 +++++
 drivers/net/ethernet/huawei/hinic/hinic_hw_csr.h   |  25 +++
 3 files changed, 235 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.c b/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.c
index b2f6bd8..874a262 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.c
@@ -13,6 +13,7 @@
  *
  */
 
+#include <linux/kernel.h>
 #include <linux/types.h>
 #include <linux/errno.h>
 #include <linux/pci.h>
@@ -21,8 +22,12 @@
 #include <linux/dma-mapping.h>
 #include <linux/bitops.h>
 #include <linux/err.h>
+#include <linux/jiffies.h>
+#include <linux/delay.h>
+#include <linux/log2.h>
 #include <asm/byteorder.h>
 
+#include "hinic_hw_csr.h"
 #include "hinic_hw_if.h"
 #include "hinic_hw_api_cmd.h"
 
@@ -35,8 +40,157 @@
 		(((cell_size) >= API_CMD_CELL_SIZE_MIN) ? \
 		 (1 << (fls(cell_size - 1))) : API_CMD_CELL_SIZE_MIN)
 
+#define API_CMD_CELL_SIZE_VAL(size)             \
+		ilog2((size) >> API_CMD_CELL_SIZE_SHIFT)
+
 #define API_CMD_BUF_SIZE                        2048
 
+#define API_CMD_TIMEOUT                 1000
+
+enum api_cmd_xor_chk_level {
+	XOR_CHK_DIS = 0,
+
+	XOR_CHK_ALL = 3,
+};
+
+/**
+ * api_cmd_hw_restart - restart the chain in the HW
+ * @chain: the API CMD specific chain to restart
+ *
+ * Return 0 - Success, negative - Failure
+ **/
+static int api_cmd_hw_restart(struct hinic_api_cmd_chain *chain)
+{
+	struct hinic_hwif *hwif = chain->hwif;
+	unsigned long end;
+	u32 reg_addr, val;
+	int err = -ETIMEDOUT;
+
+	/* Read Modify Write */
+	reg_addr = HINIC_CSR_API_CMD_CHAIN_REQ_ADDR(chain->chain_type);
+	val = hinic_hwif_read_reg(hwif, reg_addr);
+
+	val = HINIC_API_CMD_CHAIN_REQ_CLEAR(val, RESTART);
+	val |= HINIC_API_CMD_CHAIN_REQ_SET(1, RESTART);
+
+	hinic_hwif_write_reg(hwif, reg_addr, val);
+
+	end = jiffies + msecs_to_jiffies(API_CMD_TIMEOUT);
+	do {
+		val = hinic_hwif_read_reg(hwif, reg_addr);
+
+		if (!HINIC_API_CMD_CHAIN_REQ_GET(val, RESTART)) {
+			err = 0;
+			break;
+		}
+
+		msleep(20);
+	} while (time_before(jiffies, end));
+
+	return err;
+}
+
+/**
+ * api_cmd_ctrl_init - set the control register of a chain
+ * @chain: the API CMD specific chain to set control register for
+ **/
+static void api_cmd_ctrl_init(struct hinic_api_cmd_chain *chain)
+{
+	struct hinic_hwif *hwif = chain->hwif;
+	u32 reg_addr, ctrl;
+	u16 cell_size;
+
+	/* Read Modify Write */
+	reg_addr = HINIC_CSR_API_CMD_CHAIN_CTRL_ADDR(chain->chain_type);
+
+	cell_size = API_CMD_CELL_SIZE_VAL(chain->cell_size);
+
+	ctrl = hinic_hwif_read_reg(hwif, reg_addr);
+
+	ctrl =  HINIC_API_CMD_CHAIN_CTRL_CLEAR(ctrl, RESTART_WB_STAT) &
+		HINIC_API_CMD_CHAIN_CTRL_CLEAR(ctrl, XOR_ERR)   &
+		HINIC_API_CMD_CHAIN_CTRL_CLEAR(ctrl, AEQE_EN)   &
+		HINIC_API_CMD_CHAIN_CTRL_CLEAR(ctrl, XOR_CHK_EN) &
+		HINIC_API_CMD_CHAIN_CTRL_CLEAR(ctrl, CELL_SIZE);
+
+	ctrl |= HINIC_API_CMD_CHAIN_CTRL_SET(1, XOR_ERR) |
+		HINIC_API_CMD_CHAIN_CTRL_SET(XOR_CHK_ALL, XOR_CHK_EN) |
+		HINIC_API_CMD_CHAIN_CTRL_SET(cell_size, CELL_SIZE);
+
+	hinic_hwif_write_reg(hwif, reg_addr, ctrl);
+}
+
+/**
+ * api_cmd_set_status_addr - set the status address of a chain in the HW
+ * @chain: the API CMD specific chain to set in HW status address for
+ **/
+static void api_cmd_set_status_addr(struct hinic_api_cmd_chain *chain)
+{
+	struct hinic_hwif *hwif = chain->hwif;
+	u32 addr, val;
+
+	addr = HINIC_CSR_API_CMD_STATUS_HI_ADDR(chain->chain_type);
+	val = upper_32_bits(chain->wb_status_paddr);
+	hinic_hwif_write_reg(hwif, addr, val);
+
+	addr = HINIC_CSR_API_CMD_STATUS_LO_ADDR(chain->chain_type);
+	val = lower_32_bits(chain->wb_status_paddr);
+	hinic_hwif_write_reg(hwif, addr, val);
+}
+
+/**
+ * api_cmd_set_num_cells - set the number cells of a chain in the HW
+ * @chain: the API CMD specific chain to set in HW the number of cells for
+ **/
+static void api_cmd_set_num_cells(struct hinic_api_cmd_chain *chain)
+{
+	struct hinic_hwif *hwif = chain->hwif;
+	u32 addr, val;
+
+	addr = HINIC_CSR_API_CMD_CHAIN_NUM_CELLS_ADDR(chain->chain_type);
+	val  = chain->num_cells;
+	hinic_hwif_write_reg(hwif, addr, val);
+}
+
+/**
+ * api_cmd_head_init - set the head of a chain in the HW
+ * @chain: the API CMD specific chain to set in HW the head for
+ **/
+static void api_cmd_head_init(struct hinic_api_cmd_chain *chain)
+{
+	struct hinic_hwif *hwif = chain->hwif;
+	u32 addr, val;
+
+	addr = HINIC_CSR_API_CMD_CHAIN_HEAD_HI_ADDR(chain->chain_type);
+	val = upper_32_bits(chain->head_cell_paddr);
+	hinic_hwif_write_reg(hwif, addr, val);
+
+	addr = HINIC_CSR_API_CMD_CHAIN_HEAD_LO_ADDR(chain->chain_type);
+	val = lower_32_bits(chain->head_cell_paddr);
+	hinic_hwif_write_reg(hwif, addr, val);
+}
+
+/**
+ * api_cmd_chain_hw_clean - clean the HW
+ * @chain: the API CMD specific chain
+ **/
+static void api_cmd_chain_hw_clean(struct hinic_api_cmd_chain *chain)
+{
+	struct hinic_hwif *hwif = chain->hwif;
+	u32 addr, ctrl;
+
+	addr = HINIC_CSR_API_CMD_CHAIN_CTRL_ADDR(chain->chain_type);
+
+	ctrl = hinic_hwif_read_reg(hwif, addr);
+	ctrl = HINIC_API_CMD_CHAIN_CTRL_CLEAR(ctrl, RESTART_WB_STAT) &
+		HINIC_API_CMD_CHAIN_CTRL_CLEAR(ctrl, XOR_ERR)   &
+		HINIC_API_CMD_CHAIN_CTRL_CLEAR(ctrl, AEQE_EN)   &
+		HINIC_API_CMD_CHAIN_CTRL_CLEAR(ctrl, XOR_CHK_EN) &
+		HINIC_API_CMD_CHAIN_CTRL_CLEAR(ctrl, CELL_SIZE);
+
+	hinic_hwif_write_reg(hwif, addr, ctrl);
+}
+
 /**
  * api_cmd_chain_hw_init - initialize the chain in the HW
  * @chain: the API CMD specific chain to initialize in HW
@@ -45,7 +199,23 @@
  **/
 static int api_cmd_chain_hw_init(struct hinic_api_cmd_chain *chain)
 {
-	/* should be implemented */
+	struct hinic_hwif *hwif = chain->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+	int err;
+
+	api_cmd_chain_hw_clean(chain);
+
+	api_cmd_set_status_addr(chain);
+
+	err = api_cmd_hw_restart(chain);
+	if (err) {
+		dev_err(&pdev->dev, "Failed to restart API CMD HW\n");
+		return err;
+	}
+
+	api_cmd_ctrl_init(chain);
+	api_cmd_set_num_cells(chain);
+	api_cmd_head_init(chain);
 	return 0;
 }
 
@@ -373,6 +543,7 @@ static void api_chain_free(struct hinic_api_cmd_chain *chain)
  **/
 static void api_cmd_destroy_chain(struct hinic_api_cmd_chain *chain)
 {
+	api_cmd_chain_hw_clean(chain);
 	api_cmd_destroy_cells(chain, chain->num_cells);
 	api_chain_free(chain);
 }
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.h b/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.h
index 9c7d1e5..0c83b80 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.h
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.h
@@ -20,6 +20,44 @@
 
 #include "hinic_hw_if.h"
 
+#define HINIC_API_CMD_CHAIN_REQ_RESTART_SHIFT                   1
+
+#define HINIC_API_CMD_CHAIN_REQ_RESTART_MASK                    0x1
+
+#define HINIC_API_CMD_CHAIN_REQ_SET(val, member)                \
+	(((u32)(val) & HINIC_API_CMD_CHAIN_REQ_##member##_MASK) << \
+	 HINIC_API_CMD_CHAIN_REQ_##member##_SHIFT)
+
+#define HINIC_API_CMD_CHAIN_REQ_GET(val, member)                \
+	(((val) >> HINIC_API_CMD_CHAIN_REQ_##member##_SHIFT) &  \
+	 HINIC_API_CMD_CHAIN_REQ_##member##_MASK)
+
+#define HINIC_API_CMD_CHAIN_REQ_CLEAR(val, member)              \
+	((val) & (~(HINIC_API_CMD_CHAIN_REQ_##member##_MASK     \
+	 << HINIC_API_CMD_CHAIN_REQ_##member##_SHIFT)))
+
+#define HINIC_API_CMD_CHAIN_CTRL_RESTART_WB_STAT_SHIFT          1
+#define HINIC_API_CMD_CHAIN_CTRL_XOR_ERR_SHIFT                  2
+#define HINIC_API_CMD_CHAIN_CTRL_AEQE_EN_SHIFT                  4
+#define HINIC_API_CMD_CHAIN_CTRL_AEQ_ID_SHIFT                   8
+#define HINIC_API_CMD_CHAIN_CTRL_XOR_CHK_EN_SHIFT               28
+#define HINIC_API_CMD_CHAIN_CTRL_CELL_SIZE_SHIFT                30
+
+#define HINIC_API_CMD_CHAIN_CTRL_RESTART_WB_STAT_MASK           0x1
+#define HINIC_API_CMD_CHAIN_CTRL_XOR_ERR_MASK                   0x1
+#define HINIC_API_CMD_CHAIN_CTRL_AEQE_EN_MASK                   0x1
+#define HINIC_API_CMD_CHAIN_CTRL_AEQ_ID_MASK                    0x3
+#define HINIC_API_CMD_CHAIN_CTRL_XOR_CHK_EN_MASK                0x3
+#define HINIC_API_CMD_CHAIN_CTRL_CELL_SIZE_MASK                 0x3
+
+#define HINIC_API_CMD_CHAIN_CTRL_SET(val, member)               \
+	(((u32)(val) & HINIC_API_CMD_CHAIN_CTRL_##member##_MASK) << \
+	 HINIC_API_CMD_CHAIN_CTRL_##member##_SHIFT)
+
+#define HINIC_API_CMD_CHAIN_CTRL_CLEAR(val, member)             \
+	((val) & (~(HINIC_API_CMD_CHAIN_CTRL_##member##_MASK    \
+	 << HINIC_API_CMD_CHAIN_CTRL_##member##_SHIFT)))
+
 enum hinic_api_cmd_chain_type {
 	HINIC_API_CMD_WRITE_TO_MGMT_CPU = 2,
 
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_csr.h b/drivers/net/ethernet/huawei/hinic/hinic_hw_csr.h
index c3440a9..0c100ee 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_csr.h
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_csr.h
@@ -33,4 +33,29 @@
 #define HINIC_CSR_PPF_ELECTION_ADDR(idx)                \
 	(HINIC_ELECTION_BASE +  (idx) * HINIC_PPF_ELECTION_STRIDE)
 
+/* API CMD registers */
+#define HINIC_CSR_API_CMD_BASE                          0xF000
+
+#define HINIC_CSR_API_CMD_STRIDE                        0x100
+
+#define HINIC_CSR_API_CMD_CHAIN_HEAD_HI_ADDR(idx)       \
+	(HINIC_CSR_API_CMD_BASE + 0x0 + (idx) * HINIC_CSR_API_CMD_STRIDE)
+
+#define HINIC_CSR_API_CMD_CHAIN_HEAD_LO_ADDR(idx)       \
+	(HINIC_CSR_API_CMD_BASE + 0x4 + (idx) * HINIC_CSR_API_CMD_STRIDE)
+
+#define HINIC_CSR_API_CMD_STATUS_HI_ADDR(idx)           \
+	(HINIC_CSR_API_CMD_BASE + 0x8 + (idx) * HINIC_CSR_API_CMD_STRIDE)
+
+#define HINIC_CSR_API_CMD_STATUS_LO_ADDR(idx)           \
+	(HINIC_CSR_API_CMD_BASE + 0xC + (idx) * HINIC_CSR_API_CMD_STRIDE)
+
+#define HINIC_CSR_API_CMD_CHAIN_NUM_CELLS_ADDR(idx)     \
+	(HINIC_CSR_API_CMD_BASE + 0x10 + (idx) * HINIC_CSR_API_CMD_STRIDE)
+
+#define HINIC_CSR_API_CMD_CHAIN_CTRL_ADDR(idx)          \
+	(HINIC_CSR_API_CMD_BASE + 0x14 + (idx) * HINIC_CSR_API_CMD_STRIDE)
+
+#define HINIC_CSR_API_CMD_CHAIN_REQ_ADDR(idx)           \
+	(HINIC_CSR_API_CMD_BASE + 0x20 + (idx) * HINIC_CSR_API_CMD_STRIDE)
 #endif
-- 
1.9.1

[toc] | [prev] | [next] | [standalone]


#1702877 — [PATCH V3 net-next 20/21] net-next/hinic: Add ethtool and stats

FromAviad Krawczyk <aviad.krawczyk@huawei.com>
Date2017-08-03 12:10 +0200
Subject[PATCH V3 net-next 20/21] net-next/hinic: Add ethtool and stats
Message-ID<ualeP-6Uy-17@gated-at.bofh.it>
In reply to#1702870
Add ethtool operations and statistics operations.

Signed-off-by: Aviad Krawczyk <aviad.krawczyk@huawei.com>
Signed-off-by: Zhao Chen <zhaochen6@huawei.com>
---
 drivers/net/ethernet/huawei/hinic/hinic_dev.h  |   3 +
 drivers/net/ethernet/huawei/hinic/hinic_main.c | 220 ++++++++++++++++++++++++-
 drivers/net/ethernet/huawei/hinic/hinic_port.c |  29 ++++
 drivers/net/ethernet/huawei/hinic/hinic_port.h |  45 +++++
 drivers/net/ethernet/huawei/hinic/hinic_rx.c   |  19 +++
 drivers/net/ethernet/huawei/hinic/hinic_rx.h   |   2 +
 drivers/net/ethernet/huawei/hinic/hinic_tx.c   |  22 +++
 drivers/net/ethernet/huawei/hinic/hinic_tx.h   |   2 +
 8 files changed, 341 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/huawei/hinic/hinic_dev.h b/drivers/net/ethernet/huawei/hinic/hinic_dev.h
index 15d0c2e..5186cc9 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_dev.h
+++ b/drivers/net/ethernet/huawei/hinic/hinic_dev.h
@@ -56,6 +56,9 @@ struct hinic_dev {
 
 	struct hinic_txq                *txqs;
 	struct hinic_rxq                *rxqs;
+
+	struct hinic_txq_stats          tx_stats;
+	struct hinic_rxq_stats          rx_stats;
 };
 
 #endif
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_main.c b/drivers/net/ethernet/huawei/hinic/hinic_main.c
index 3eb3fc1..4e0709a 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_main.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_main.c
@@ -69,6 +69,180 @@
 
 static int change_mac_addr(struct net_device *netdev, const u8 *addr);
 
+static int hinic_get_link_ksettings(struct net_device *netdev,
+				    struct ethtool_link_ksettings
+				    *link_ksettings)
+{
+	struct hinic_dev *nic_dev = netdev_priv(netdev);
+	struct hinic_port_cap port_cap;
+	enum hinic_autoneg_cap autoneg_cap;
+	enum hinic_autoneg_state autoneg_state;
+	enum hinic_port_link_state link_state;
+	int err;
+
+	ethtool_link_ksettings_zero_link_mode(link_ksettings, advertising);
+	ethtool_link_ksettings_add_link_mode(link_ksettings, supported,
+					     Autoneg);
+
+	link_ksettings->base.speed = SPEED_UNKNOWN;
+	link_ksettings->base.autoneg = AUTONEG_DISABLE;
+	link_ksettings->base.duplex = DUPLEX_UNKNOWN;
+
+	err = hinic_port_get_cap(nic_dev, &port_cap);
+	if (err) {
+		netif_err(nic_dev, drv, netdev,
+			  "Failed to get port capabilities\n");
+		return err;
+	}
+
+	err = hinic_port_link_state(nic_dev, &link_state);
+	if (err) {
+		netif_err(nic_dev, drv, netdev,
+			  "Failed to get port link state\n");
+		return err;
+	}
+
+	if (link_state != HINIC_LINK_STATE_UP) {
+		netif_info(nic_dev, drv, netdev, "No link\n");
+		return err;
+	}
+
+	switch (port_cap.speed) {
+	case HINIC_SPEED_10MB_LINK:
+		link_ksettings->base.speed = SPEED_10;
+		break;
+
+	case HINIC_SPEED_100MB_LINK:
+		link_ksettings->base.speed = SPEED_100;
+		break;
+
+	case HINIC_SPEED_1000MB_LINK:
+		link_ksettings->base.speed = SPEED_1000;
+		break;
+
+	case HINIC_SPEED_10GB_LINK:
+		link_ksettings->base.speed = SPEED_10000;
+		break;
+
+	case HINIC_SPEED_25GB_LINK:
+		link_ksettings->base.speed = SPEED_25000;
+		break;
+
+	case HINIC_SPEED_40GB_LINK:
+		link_ksettings->base.speed = SPEED_40000;
+		break;
+
+	case HINIC_SPEED_100GB_LINK:
+		link_ksettings->base.speed = SPEED_100000;
+		break;
+
+	default:
+		link_ksettings->base.speed = SPEED_UNKNOWN;
+		break;
+	}
+
+	autoneg_cap = port_cap.autoneg_cap;
+	autoneg_state = port_cap.autoneg_state;
+
+	if (!!(autoneg_cap & HINIC_AUTONEG_SUPPORTED))
+		ethtool_link_ksettings_add_link_mode(link_ksettings,
+						     advertising, Autoneg);
+
+	link_ksettings->base.autoneg = (autoneg_state == HINIC_AUTONEG_ACTIVE) ?
+				       AUTONEG_ENABLE : AUTONEG_DISABLE;
+	link_ksettings->base.duplex = (port_cap.duplex == HINIC_DUPLEX_FULL) ?
+				      DUPLEX_FULL : DUPLEX_HALF;
+
+	return 0;
+}
+
+static void hinic_get_drvinfo(struct net_device *netdev,
+			      struct ethtool_drvinfo *info)
+{
+	struct hinic_dev *nic_dev = netdev_priv(netdev);
+	struct hinic_hwdev *hwdev = nic_dev->hwdev;
+	struct hinic_hwif *hwif = hwdev->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+
+	strlcpy(info->driver, HINIC_DRV_NAME, sizeof(info->driver));
+	strlcpy(info->bus_info, pci_name(pdev), sizeof(info->bus_info));
+}
+
+static void hinic_get_ringparam(struct net_device *netdev,
+				struct ethtool_ringparam *ring)
+{
+	ring->rx_max_pending = HINIC_RQ_DEPTH;
+	ring->tx_max_pending = HINIC_SQ_DEPTH;
+	ring->rx_pending = HINIC_RQ_DEPTH;
+	ring->tx_pending = HINIC_SQ_DEPTH;
+}
+
+static void hinic_get_channels(struct net_device *netdev,
+			       struct ethtool_channels *channels)
+{
+	struct hinic_dev *nic_dev = netdev_priv(netdev);
+	struct hinic_hwdev *hwdev = nic_dev->hwdev;
+
+	channels->max_rx = hwdev->nic_cap.max_qps;
+	channels->max_tx = hwdev->nic_cap.max_qps;
+	channels->max_other = 0;
+	channels->max_combined = 0;
+	channels->rx_count = hinic_hwdev_num_qps(hwdev);
+	channels->tx_count = hinic_hwdev_num_qps(hwdev);
+	channels->other_count = 0;
+	channels->combined_count = 0;
+}
+
+static const struct ethtool_ops hinic_ethtool_ops = {
+	.get_link_ksettings = hinic_get_link_ksettings,
+	.get_drvinfo = hinic_get_drvinfo,
+	.get_link = ethtool_op_get_link,
+	.get_ringparam = hinic_get_ringparam,
+	.get_channels = hinic_get_channels,
+};
+
+static void update_nic_stats(struct hinic_dev *nic_dev)
+{
+	struct hinic_rxq_stats *nic_rx_stats = &nic_dev->rx_stats;
+	struct hinic_txq_stats *nic_tx_stats = &nic_dev->tx_stats;
+	struct hinic_hwdev *hwdev = nic_dev->hwdev;
+	int i, num_qps = hinic_hwdev_num_qps(hwdev);
+	struct hinic_rxq_stats rx_stats;
+	struct hinic_txq_stats tx_stats;
+
+	u64_stats_init(&tx_stats.syncp);
+	u64_stats_init(&rx_stats.syncp);
+
+	for (i = 0; i < num_qps; i++) {
+		struct hinic_rxq *rxq = &nic_dev->rxqs[i];
+
+		hinic_rxq_get_stats(rxq, &rx_stats);
+
+		u64_stats_update_begin(&nic_rx_stats->syncp);
+		nic_rx_stats->bytes += rx_stats.bytes;
+		nic_rx_stats->pkts += rx_stats.pkts;
+		u64_stats_update_end(&nic_rx_stats->syncp);
+
+		hinic_rxq_clean_stats(rxq);
+	}
+
+	for (i = 0; i < num_qps; i++) {
+		struct hinic_txq *txq = &nic_dev->txqs[i];
+
+		hinic_txq_get_stats(txq, &tx_stats);
+
+		u64_stats_update_begin(&nic_tx_stats->syncp);
+		nic_tx_stats->bytes += tx_stats.bytes;
+		nic_tx_stats->pkts += tx_stats.pkts;
+		nic_tx_stats->tx_busy += tx_stats.tx_busy;
+		nic_tx_stats->tx_wake += tx_stats.tx_wake;
+		nic_tx_stats->tx_dropped += tx_stats.tx_dropped;
+		u64_stats_update_end(&nic_tx_stats->syncp);
+
+		hinic_txq_clean_stats(txq);
+	}
+}
+
 /**
  * create_txqs - Create the Logical Tx Queues of specific NIC device
  * @nic_dev: the specific NIC device
@@ -308,6 +482,8 @@ static int hinic_close(struct net_device *netdev)
 	netif_carrier_off(netdev);
 	netif_tx_disable(netdev);
 
+	update_nic_stats(nic_dev);
+
 	up(&nic_dev->mgmt_lock);
 
 	err = hinic_port_set_func_state(nic_dev, HINIC_FUNC_PORT_DISABLE);
@@ -588,6 +764,39 @@ static void hinic_tx_timeout(struct net_device *netdev)
 	netif_err(nic_dev, drv, netdev, "Tx timeout\n");
 }
 
+static void hinic_get_stats64(struct net_device *netdev,
+			      struct rtnl_link_stats64 *stats)
+{
+	struct hinic_dev *nic_dev = netdev_priv(netdev);
+	struct hinic_rxq_stats *nic_rx_stats = &nic_dev->rx_stats;
+	struct hinic_txq_stats *nic_tx_stats = &nic_dev->tx_stats;
+
+	stats->rx_bytes = nic_rx_stats->bytes;
+	stats->rx_packets = nic_rx_stats->pkts;
+
+	stats->tx_bytes = nic_tx_stats->bytes;
+	stats->tx_packets = nic_tx_stats->pkts;
+	stats->tx_errors = nic_tx_stats->tx_dropped;
+
+	down(&nic_dev->mgmt_lock);
+
+	if (!(nic_dev->flags & HINIC_INTF_UP)) {
+		up(&nic_dev->mgmt_lock);
+		return;
+	}
+
+	update_nic_stats(nic_dev);
+
+	up(&nic_dev->mgmt_lock);
+
+	stats->rx_bytes = nic_rx_stats->bytes;
+	stats->rx_packets = nic_rx_stats->pkts;
+
+	stats->tx_bytes = nic_tx_stats->bytes;
+	stats->tx_packets = nic_tx_stats->pkts;
+	stats->tx_errors = nic_tx_stats->tx_dropped;
+}
+
 static const struct net_device_ops hinic_netdev_ops = {
 	.ndo_open = hinic_open,
 	.ndo_stop = hinic_close,
@@ -599,7 +808,7 @@ static void hinic_tx_timeout(struct net_device *netdev)
 	.ndo_set_rx_mode = hinic_set_rx_mode,
 	.ndo_start_xmit = hinic_xmit_frame,
 	.ndo_tx_timeout = hinic_tx_timeout,
-	/* more operations should be filled */
+	.ndo_get_stats64 = hinic_get_stats64,
 };
 
 static void netdev_features_init(struct net_device *netdev)
@@ -674,6 +883,8 @@ static int nic_dev_init(struct pci_dev *pdev)
 	struct hinic_dev *nic_dev;
 	struct net_device *netdev;
 	struct hinic_hwdev *hwdev;
+	struct hinic_txq_stats *tx_stats;
+	struct hinic_rxq_stats *rx_stats;
 	struct hinic_rx_mode_work *rx_mode_work;
 	int err, num_qps;
 
@@ -698,6 +909,7 @@ static int nic_dev_init(struct pci_dev *pdev)
 	}
 
 	netdev->netdev_ops = &hinic_netdev_ops;
+	netdev->ethtool_ops = &hinic_ethtool_ops;
 
 	nic_dev = (struct hinic_dev *)netdev_priv(netdev);
 	nic_dev->netdev = netdev;
@@ -711,6 +923,12 @@ static int nic_dev_init(struct pci_dev *pdev)
 
 	sema_init(&nic_dev->mgmt_lock, 1);
 
+	tx_stats = &nic_dev->tx_stats;
+	rx_stats = &nic_dev->rx_stats;
+
+	u64_stats_init(&tx_stats->syncp);
+	u64_stats_init(&rx_stats->syncp);
+
 	nic_dev->vlan_bitmap = devm_kzalloc(&pdev->dev,
 					    VLAN_BITMAP_SIZE(nic_dev),
 					    GFP_KERNEL);
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_port.c b/drivers/net/ethernet/huawei/hinic/hinic_port.c
index 9c179b6..ed0c05b 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_port.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_port.c
@@ -372,3 +372,32 @@ int hinic_port_set_func_state(struct hinic_dev *nic_dev,
 
 	return 0;
 }
+
+/**
+ * hinic_port_get_cap - get port capabilities
+ * @nic_dev: nic device
+ * @port_cap: returned port capabilities
+ *
+ * Return 0 - Success, negative - Failure
+ **/
+int hinic_port_get_cap(struct hinic_dev *nic_dev,
+		       struct hinic_port_cap *port_cap)
+{
+	struct hinic_hwdev *hwdev = nic_dev->hwdev;
+	struct hinic_hwif *hwif = hwdev->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+	u16 out_size;
+	int err;
+
+	port_cap->func_idx = HINIC_HWIF_FUNC_IDX(hwif);
+
+	err = hinic_port_msg_cmd(hwdev, HINIC_PORT_CMD_GET_CAP,
+				 port_cap, sizeof(*port_cap),
+				 port_cap, &out_size);
+	if (err || (out_size != sizeof(*port_cap)) || port_cap->status) {
+		dev_err(&pdev->dev, "Failed to get port capabilities\n");
+		return -EINVAL;
+	}
+
+	return 0;
+}
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_port.h b/drivers/net/ethernet/huawei/hinic/hinic_port.h
index 77e243b..6def531 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_port.h
+++ b/drivers/net/ethernet/huawei/hinic/hinic_port.h
@@ -45,6 +45,33 @@ enum hinic_func_port_state {
 	HINIC_FUNC_PORT_ENABLE  = 2,
 };
 
+enum hinic_autoneg_cap {
+	HINIC_AUTONEG_UNSUPPORTED,
+	HINIC_AUTONEG_SUPPORTED,
+};
+
+enum hinic_autoneg_state {
+	HINIC_AUTONEG_DISABLED,
+	HINIC_AUTONEG_ACTIVE,
+};
+
+enum hinic_duplex {
+	HINIC_DUPLEX_HALF,
+	HINIC_DUPLEX_FULL,
+};
+
+enum hinic_speed {
+	HINIC_SPEED_10MB_LINK = 0,
+	HINIC_SPEED_100MB_LINK,
+	HINIC_SPEED_1000MB_LINK,
+	HINIC_SPEED_10GB_LINK,
+	HINIC_SPEED_25GB_LINK,
+	HINIC_SPEED_40GB_LINK,
+	HINIC_SPEED_100GB_LINK,
+
+	HINIC_SPEED_UNKNOWN = 0xFF,
+};
+
 struct hinic_port_mac_cmd {
 	u8              status;
 	u8              version;
@@ -125,6 +152,21 @@ struct hinic_port_func_state_cmd {
 	u8      rsvd2[3];
 };
 
+struct hinic_port_cap {
+	u8      status;
+	u8      version;
+	u8      rsvd0[6];
+
+	u16     func_idx;
+	u16     rsvd1;
+	u8      port_type;
+	u8      autoneg_cap;
+	u8      autoneg_state;
+	u8      duplex;
+	u8      speed;
+	u8      resv2[3];
+};
+
 int hinic_port_add_mac(struct hinic_dev *nic_dev, const u8 *addr,
 		       u16 vlan_id);
 
@@ -150,4 +192,7 @@ int hinic_port_set_state(struct hinic_dev *nic_dev,
 int hinic_port_set_func_state(struct hinic_dev *nic_dev,
 			      enum hinic_func_port_state state);
 
+int hinic_port_get_cap(struct hinic_dev *nic_dev,
+		       struct hinic_port_cap *port_cap);
+
 #endif
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_rx.c b/drivers/net/ethernet/huawei/hinic/hinic_rx.c
index d77defa..1398678 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_rx.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_rx.c
@@ -58,6 +58,25 @@ void hinic_rxq_clean_stats(struct hinic_rxq *rxq)
 }
 
 /**
+ * hinic_rxq_get_stats - get statistics of Rx Queue
+ * @rxq: Logical Rx Queue
+ * @stats: return updated stats here
+ **/
+void hinic_rxq_get_stats(struct hinic_rxq *rxq, struct hinic_rxq_stats *stats)
+{
+	struct hinic_rxq_stats *rxq_stats = &rxq->rxq_stats;
+	unsigned int start;
+
+	u64_stats_update_begin(&stats->syncp);
+	do {
+		start = u64_stats_fetch_begin(&rxq_stats->syncp);
+		stats->pkts = rxq_stats->pkts;
+		stats->bytes = rxq_stats->bytes;
+	} while (u64_stats_fetch_retry(&rxq_stats->syncp, start));
+	u64_stats_update_end(&stats->syncp);
+}
+
+/**
  * rxq_stats_init - Initialize the statistics of specific queue
  * @rxq: Logical Rx Queue
  **/
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_rx.h b/drivers/net/ethernet/huawei/hinic/hinic_rx.h
index 538c886..27c9af4 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_rx.h
+++ b/drivers/net/ethernet/huawei/hinic/hinic_rx.h
@@ -45,6 +45,8 @@ struct hinic_rxq {
 
 void hinic_rxq_clean_stats(struct hinic_rxq *rxq);
 
+void hinic_rxq_get_stats(struct hinic_rxq *rxq, struct hinic_rxq_stats *stats);
+
 int hinic_init_rxq(struct hinic_rxq *rxq, struct hinic_rq *rq,
 		   struct net_device *netdev);
 
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_tx.c b/drivers/net/ethernet/huawei/hinic/hinic_tx.c
index c3e36ab..02d31b2 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_tx.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_tx.c
@@ -67,6 +67,28 @@ void hinic_txq_clean_stats(struct hinic_txq *txq)
 }
 
 /**
+ * hinic_txq_get_stats - get statistics of Tx Queue
+ * @txq: Logical Tx Queue
+ * @stats: return updated stats here
+ **/
+void hinic_txq_get_stats(struct hinic_txq *txq, struct hinic_txq_stats *stats)
+{
+	struct hinic_txq_stats *txq_stats = &txq->txq_stats;
+	unsigned int start;
+
+	u64_stats_update_begin(&stats->syncp);
+	do {
+		start = u64_stats_fetch_begin(&txq_stats->syncp);
+		stats->pkts = txq_stats->pkts;
+		stats->bytes = txq_stats->bytes;
+		stats->tx_busy = txq_stats->tx_busy;
+		stats->tx_wake = txq_stats->tx_wake;
+		stats->tx_dropped = txq_stats->tx_dropped;
+	} while (u64_stats_fetch_retry(&txq_stats->syncp, start));
+	u64_stats_update_end(&stats->syncp);
+}
+
+/**
  * txq_stats_init - Initialize the statistics of specific queue
  * @txq: Logical Tx Queue
  **/
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_tx.h b/drivers/net/ethernet/huawei/hinic/hinic_tx.h
index 7123c7f..1fa55dc 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_tx.h
+++ b/drivers/net/ethernet/huawei/hinic/hinic_tx.h
@@ -50,6 +50,8 @@ struct hinic_txq {
 
 void hinic_txq_clean_stats(struct hinic_txq *txq);
 
+void hinic_txq_get_stats(struct hinic_txq *txq, struct hinic_txq_stats *stats);
+
 netdev_tx_t hinic_xmit_frame(struct sk_buff *skb, struct net_device *netdev);
 
 int hinic_init_txq(struct hinic_txq *txq, struct hinic_sq *sq,
-- 
1.9.1

[toc] | [prev] | [next] | [standalone]


#1702887 — [PATCH V3 net-next 19/21] net-next/hinic: Add Tx operation

FromAviad Krawczyk <aviad.krawczyk@huawei.com>
Date2017-08-03 12:10 +0200
Subject[PATCH V3 net-next 19/21] net-next/hinic: Add Tx operation
Message-ID<ualeP-6Uy-25@gated-at.bofh.it>
In reply to#1702870
Add transmit operation for sending data by qp operations.

Signed-off-by: Aviad Krawczyk <aviad.krawczyk@huawei.com>
Signed-off-by: Zhao Chen <zhaochen6@huawei.com>
---
 drivers/net/ethernet/huawei/hinic/hinic_dev.h     |   1 +
 drivers/net/ethernet/huawei/hinic/hinic_hw_dev.c  |  47 +++
 drivers/net/ethernet/huawei/hinic/hinic_hw_dev.h  |  22 ++
 drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.h |   2 +
 drivers/net/ethernet/huawei/hinic/hinic_hw_qp.c   | 243 +++++++++++++
 drivers/net/ethernet/huawei/hinic/hinic_hw_qp.h   |  47 +++
 drivers/net/ethernet/huawei/hinic/hinic_main.c    |  12 +-
 drivers/net/ethernet/huawei/hinic/hinic_tx.c      | 419 ++++++++++++++++++++++
 drivers/net/ethernet/huawei/hinic/hinic_tx.h      |  11 +
 9 files changed, 802 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/huawei/hinic/hinic_dev.h b/drivers/net/ethernet/huawei/hinic/hinic_dev.h
index 3d0f6cf..15d0c2e 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_dev.h
+++ b/drivers/net/ethernet/huawei/hinic/hinic_dev.h
@@ -43,6 +43,7 @@ struct hinic_dev {
 	struct hinic_hwdev              *hwdev;
 
 	u32                             msg_enable;
+	unsigned int                    tx_weight;
 	unsigned int                    rx_weight;
 
 	unsigned int                    flags;
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_dev.c b/drivers/net/ethernet/huawei/hinic/hinic_hw_dev.c
index 18199aa..de4a59f 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_dev.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_dev.c
@@ -40,6 +40,8 @@
 #define MAX_IRQS(max_qps, num_aeqs, num_ceqs)   \
 		 (2 * (max_qps) + (num_aeqs) + (num_ceqs))
 
+#define ADDR_IN_4BYTES(addr)            ((addr) >> 2)
+
 enum intr_type {
 	INTR_MSIX_TYPE,
 };
@@ -1003,3 +1005,48 @@ int hinic_hwdev_msix_set(struct hinic_hwdev *hwdev, u16 msix_index,
 				   lli_timer_cfg, lli_credit_limit,
 				   resend_timer);
 }
+
+/**
+ * hinic_hwdev_hw_ci_addr_set - set cons idx addr and attributes in HW for sq
+ * @hwdev: the NIC HW device
+ * @sq: send queue
+ * @pending_limit: the maximum pending update ci events (unit 8)
+ * @coalesc_timer: coalesc period for update ci (unit 8 us)
+ *
+ * Return 0 - Success, negative - Failure
+ **/
+int hinic_hwdev_hw_ci_addr_set(struct hinic_hwdev *hwdev, struct hinic_sq *sq,
+			       u8 pending_limit, u8 coalesc_timer)
+{
+	struct hinic_hwif *hwif = hwdev->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+	struct hinic_qp *qp = container_of(sq, struct hinic_qp, sq);
+	struct hinic_pfhwdev *pfhwdev;
+	struct hinic_cmd_hw_ci hw_ci;
+
+	if (!HINIC_IS_PF(hwif) && !HINIC_IS_PPF(hwif)) {
+		dev_err(&pdev->dev, "Unsupported PCI Function type\n");
+		return -EINVAL;
+	}
+
+	hw_ci.dma_attr_off  = 0;
+	hw_ci.pending_limit = pending_limit;
+	hw_ci.coalesc_timer  = coalesc_timer;
+
+	hw_ci.msix_en = 1;
+	hw_ci.msix_entry_idx = sq->msix_entry;
+
+	hw_ci.func_idx = HINIC_HWIF_FUNC_IDX(hwif);
+
+	hw_ci.sq_id = qp->q_id;
+
+	hw_ci.ci_addr = ADDR_IN_4BYTES(sq->hw_ci_dma_addr);
+
+	pfhwdev = container_of(hwdev, struct hinic_pfhwdev, hwdev);
+
+	return hinic_msg_to_mgmt(&pfhwdev->pf_to_mgmt,
+				 HINIC_MOD_COMM,
+				 HINIC_COMM_CMD_SQ_HI_CI_SET,
+				 &hw_ci, sizeof(hw_ci), NULL,
+				 NULL, HINIC_MGMT_MSG_SYNC);
+}
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_dev.h b/drivers/net/ethernet/huawei/hinic/hinic_hw_dev.h
index e7277d1..0f5563f 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_dev.h
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_dev.h
@@ -153,6 +153,25 @@ struct hinic_cmd_base_qpn {
 	u16     qpn;
 };
 
+struct hinic_cmd_hw_ci {
+	u8      status;
+	u8      version;
+	u8      rsvd0[6];
+
+	u16     func_idx;
+
+	u8      dma_attr_off;
+	u8      pending_limit;
+	u8      coalesc_timer;
+
+	u8      msix_en;
+	u16     msix_entry_idx;
+
+	u32     sq_id;
+	u32     rsvd1;
+	u64     ci_addr;
+};
+
 struct hinic_hwdev {
 	struct hinic_hwif               *hwif;
 	struct msix_entry               *msix_entries;
@@ -214,4 +233,7 @@ int hinic_hwdev_msix_set(struct hinic_hwdev *hwdev, u16 msix_index,
 			 u8 lli_timer_cfg, u8 lli_credit_limit,
 			 u8 resend_timer);
 
+int hinic_hwdev_hw_ci_addr_set(struct hinic_hwdev *hwdev, struct hinic_sq *sq,
+			       u8 pending_limit, u8 coalesc_timer);
+
 #endif
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.h b/drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.h
index b99af77..320711e 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.h
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.h
@@ -77,6 +77,8 @@ enum hinic_comm_cmd {
 	HINIC_COMM_CMD_HWCTXT_SET       = 0x12,
 	HINIC_COMM_CMD_HWCTXT_GET       = 0x13,
 
+	HINIC_COMM_CMD_SQ_HI_CI_SET     = 0x14,
+
 	HINIC_COMM_CMD_RES_STATE_SET    = 0x24,
 
 	HINIC_COMM_CMD_IO_RES_CLEAR     = 0x29,
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_qp.c b/drivers/net/ethernet/huawei/hinic/hinic_hw_qp.c
index 717b42a..50869ce 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_qp.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_qp.c
@@ -23,6 +23,7 @@
 #include <linux/sizes.h>
 #include <linux/atomic.h>
 #include <linux/skbuff.h>
+#include <linux/io.h>
 #include <asm/barrier.h>
 #include <asm/byteorder.h>
 
@@ -32,6 +33,7 @@
 #include "hinic_hw_wq.h"
 #include "hinic_hw_qp_ctxt.h"
 #include "hinic_hw_qp.h"
+#include "hinic_hw_io.h"
 
 #define SQ_DB_OFF               SZ_2K
 
@@ -55,8 +57,24 @@
 
 #define SIZE_16BYTES(size)      (ALIGN(size, 16) >> 4)
 #define SIZE_8BYTES(size)       (ALIGN(size, 8) >> 3)
+#define SECT_SIZE_FROM_8BYTES(size)     ((size) << 3)
+
+#define SQ_DB_PI_HI_SHIFT       8
+#define SQ_DB_PI_HI(prod_idx)   ((prod_idx) >> SQ_DB_PI_HI_SHIFT)
+
+#define SQ_DB_PI_LOW_MASK       0xFF
+#define SQ_DB_PI_LOW(prod_idx)  ((prod_idx) & SQ_DB_PI_LOW_MASK)
+
+#define SQ_DB_ADDR(sq, pi)      ((u64 *)((sq)->db_base) + SQ_DB_PI_LOW(pi))
 
 #define RQ_MASKED_IDX(rq, idx)  ((idx) & (rq)->wq->mask)
+#define SQ_MASKED_IDX(sq, idx)  ((idx) & (sq)->wq->mask)
+
+#define TX_MAX_MSS_DEFAULT      0x3E00
+
+enum sq_wqe_type {
+	SQ_NORMAL_WQE = 0,
+};
 
 enum rq_completion_fmt {
 	RQ_COMPLETE_SGE = 1
@@ -435,6 +453,19 @@ void hinic_clean_rq(struct hinic_rq *rq)
 }
 
 /**
+ * hinic_get_sq_free_wqebbs - return number of free wqebbs for use
+ * @sq: send queue
+ *
+ * Return number of free wqebbs
+ **/
+int hinic_get_sq_free_wqebbs(struct hinic_sq *sq)
+{
+	struct hinic_wq *wq = sq->wq;
+
+	return atomic_read(&wq->delta) - 1;
+}
+
+/**
  * hinic_get_rq_free_wqebbs - return number of free wqebbs for use
  * @rq: recv queue
  *
@@ -447,6 +478,218 @@ int hinic_get_rq_free_wqebbs(struct hinic_rq *rq)
 	return atomic_read(&wq->delta) - 1;
 }
 
+static void sq_prepare_ctrl(struct hinic_sq_ctrl *ctrl, u16 prod_idx,
+			    int nr_descs)
+{
+	u32 ctrl_size, task_size, bufdesc_size;
+
+	ctrl_size = SIZE_8BYTES(sizeof(struct hinic_sq_ctrl));
+	task_size = SIZE_8BYTES(sizeof(struct hinic_sq_task));
+	bufdesc_size = nr_descs * sizeof(struct hinic_sq_bufdesc);
+	bufdesc_size = SIZE_8BYTES(bufdesc_size);
+
+	ctrl->ctrl_info = HINIC_SQ_CTRL_SET(bufdesc_size, BUFDESC_SECT_LEN) |
+			HINIC_SQ_CTRL_SET(task_size, TASKSECT_LEN)      |
+			HINIC_SQ_CTRL_SET(SQ_NORMAL_WQE, DATA_FORMAT)   |
+			HINIC_SQ_CTRL_SET(ctrl_size, LEN);
+
+	ctrl->queue_info = HINIC_SQ_CTRL_SET(TX_MAX_MSS_DEFAULT,
+					     QUEUE_INFO_MSS);
+}
+
+static void sq_prepare_task(struct hinic_sq_task *task)
+{
+	task->pkt_info0 =
+		HINIC_SQ_TASK_INFO0_SET(0, L2HDR_LEN) |
+		HINIC_SQ_TASK_INFO0_SET(HINIC_L4_OFF_DISABLE, L4_OFFLOAD) |
+		HINIC_SQ_TASK_INFO0_SET(HINIC_OUTER_L3TYPE_UNKNOWN,
+					INNER_L3TYPE) |
+		HINIC_SQ_TASK_INFO0_SET(HINIC_VLAN_OFF_DISABLE,
+					VLAN_OFFLOAD) |
+		HINIC_SQ_TASK_INFO0_SET(HINIC_PKT_NOT_PARSED, PARSE_FLAG);
+
+	task->pkt_info1 =
+		HINIC_SQ_TASK_INFO1_SET(HINIC_MEDIA_UNKNOWN, MEDIA_TYPE) |
+		HINIC_SQ_TASK_INFO1_SET(0, INNER_L4_LEN) |
+		HINIC_SQ_TASK_INFO1_SET(0, INNER_L3_LEN);
+
+	task->pkt_info2 =
+		HINIC_SQ_TASK_INFO2_SET(0, TUNNEL_L4_LEN) |
+		HINIC_SQ_TASK_INFO2_SET(0, OUTER_L3_LEN) |
+		HINIC_SQ_TASK_INFO2_SET(HINIC_TUNNEL_L4TYPE_UNKNOWN,
+					TUNNEL_L4TYPE) |
+		HINIC_SQ_TASK_INFO2_SET(HINIC_OUTER_L3TYPE_UNKNOWN,
+					OUTER_L3TYPE);
+
+	task->ufo_v6_identify = 0;
+
+	task->pkt_info4 = HINIC_SQ_TASK_INFO4_SET(HINIC_L2TYPE_ETH, L2TYPE);
+
+	task->zero_pad = 0;
+}
+
+/**
+ * hinic_sq_prepare_wqe - prepare wqe before insert to the queue
+ * @sq: send queue
+ * @prod_idx: pi value
+ * @sq_wqe: wqe to prepare
+ * @sges: sges for use by the wqe for send for buf addresses
+ * @nr_sges: number of sges
+ **/
+void hinic_sq_prepare_wqe(struct hinic_sq *sq, u16 prod_idx,
+			  struct hinic_sq_wqe *sq_wqe, struct hinic_sge *sges,
+			  int nr_sges)
+{
+	int i;
+
+	sq_prepare_ctrl(&sq_wqe->ctrl, prod_idx, nr_sges);
+
+	sq_prepare_task(&sq_wqe->task);
+
+	for (i = 0; i < nr_sges; i++)
+		sq_wqe->buf_descs[i].sge = sges[i];
+}
+
+/**
+ * sq_prepare_db - prepare doorbell to write
+ * @sq: send queue
+ * @prod_idx: pi value for the doorbell
+ * @cos: cos of the doorbell
+ *
+ * Return db value
+ **/
+static u32 sq_prepare_db(struct hinic_sq *sq, u16 prod_idx, unsigned int cos)
+{
+	struct hinic_qp *qp = container_of(sq, struct hinic_qp, sq);
+	int hi_prod_idx = SQ_DB_PI_HI(SQ_MASKED_IDX(sq, prod_idx));
+
+	/* Data should be written to HW in Big Endian Format */
+	return cpu_to_be32(HINIC_SQ_DB_INFO_SET(hi_prod_idx, PI_HI)     |
+			   HINIC_SQ_DB_INFO_SET(HINIC_DB_SQ_TYPE, TYPE) |
+			   HINIC_SQ_DB_INFO_SET(HINIC_DATA_PATH, PATH)  |
+			   HINIC_SQ_DB_INFO_SET(cos, COS)               |
+			   HINIC_SQ_DB_INFO_SET(qp->q_id, QID));
+}
+
+/**
+ * hinic_sq_write_db- write doorbell
+ * @sq: send queue
+ * @prod_idx: pi value for the doorbell
+ * @cos: cos of the doorbell
+ **/
+void hinic_sq_write_db(struct hinic_sq *sq, u16 prod_idx, unsigned int cos)
+{
+	u32 db_info = sq_prepare_db(sq, prod_idx, cos);
+
+	wmb();  /* Write all before the doorbell */
+
+	writel(db_info, SQ_DB_ADDR(sq, prod_idx));
+}
+
+/**
+ * hinic_sq_get_wqe - get wqe ptr in the current pi and update the pi
+ * @sq: sq to get wqe from
+ * @wqe_size: wqe size
+ * @prod_idx: returned pi
+ *
+ * Return wqe pointer
+ **/
+struct hinic_sq_wqe *hinic_sq_get_wqe(struct hinic_sq *sq,
+				      unsigned int wqe_size, u16 *prod_idx)
+{
+	struct hinic_hw_wqe *hw_wqe = hinic_get_wqe(sq->wq, wqe_size,
+						    prod_idx);
+	return &hw_wqe->sq_wqe;
+}
+
+/**
+ * hinic_sq_write_wqe - write the wqe to the sq
+ * @sq: send queue
+ * @prod_idx: pi of the wqe
+ * @sq_wqe: the wqe to write
+ * @skb: skb to save
+ * @wqe_size: the size of the wqe
+ **/
+void hinic_sq_write_wqe(struct hinic_sq *sq, u16 prod_idx,
+			struct hinic_sq_wqe *sq_wqe,
+			struct sk_buff *skb, unsigned int wqe_size)
+{
+	struct hinic_hw_wqe *hw_wqe = (struct hinic_hw_wqe *)sq_wqe;
+
+	sq->saved_skb[prod_idx] = skb;
+
+	/* The data in the HW should be in Big Endian Format */
+	hinic_cpu_to_be32(sq_wqe, wqe_size);
+
+	hinic_write_wqe(sq->wq, hw_wqe, wqe_size);
+}
+
+/**
+ * hinic_sq_read_wqe - read wqe ptr in the current ci and update the ci
+ * @sq: send queue
+ * @skb: return skb that was saved
+ * @wqe_size: the size of the wqe
+ * @cons_idx: consumer index of the wqe
+ *
+ * Return wqe in ci position
+ **/
+struct hinic_sq_wqe *hinic_sq_read_wqe(struct hinic_sq *sq,
+				       struct sk_buff **skb,
+				       unsigned int *wqe_size, u16 *cons_idx)
+{
+	struct hinic_wq *wq = sq->wq;
+	int ctrl_size = sizeof(struct hinic_sq_ctrl);
+	struct hinic_hw_wqe *hw_wqe;
+
+	/* read the ctrl section for getting wqe size */
+	hw_wqe = hinic_read_wqe(wq, ctrl_size, cons_idx);
+	if (hw_wqe) {
+		struct hinic_sq_wqe *sq_wqe = &hw_wqe->sq_wqe;
+		struct hinic_sq_ctrl *ctrl = &sq_wqe->ctrl;
+		u32 ctrl_info = be32_to_cpu(ctrl->ctrl_info);
+		int buf_sect_len = HINIC_SQ_CTRL_GET(ctrl_info,
+						     BUFDESC_SECT_LEN);
+
+		*wqe_size = sizeof(*ctrl) + sizeof(sq_wqe->task);
+		*wqe_size += SECT_SIZE_FROM_8BYTES(buf_sect_len);
+
+		*wqe_size = ALIGN(*wqe_size, wq->wqebb_size);
+		*skb = sq->saved_skb[*cons_idx];
+
+		/* using the real wqe size to read wqe again */
+		hw_wqe = hinic_read_wqe(wq, *wqe_size, cons_idx);
+	}
+
+	return &hw_wqe->sq_wqe;
+}
+
+/**
+ * hinic_sq_put_wqe - release the ci for new wqes
+ * @sq: send queue
+ * @wqe_size: the size of the wqe
+ **/
+void hinic_sq_put_wqe(struct hinic_sq *sq, unsigned int wqe_size)
+{
+	hinic_put_wqe(sq->wq, wqe_size);
+}
+
+/**
+ * hinic_sq_get_sges - get sges from the wqe
+ * @sq_wqe: wqe to get the sges from its buffer addresses
+ * @sges: returned sges
+ * @nr_sges: number sges to return
+ **/
+void hinic_sq_get_sges(struct hinic_sq_wqe *sq_wqe, struct hinic_sge *sges,
+		       int nr_sges)
+{
+	int i;
+
+	for (i = 0; i < nr_sges && i < HINIC_MAX_SQ_BUFDESCS; i++) {
+		sges[i] = sq_wqe->buf_descs[i].sge;
+		hinic_be32_to_cpu(&sges[i], sizeof(sges[i]));
+	}
+}
+
 /**
  * hinic_rq_get_wqe - get wqe ptr in the current pi and update the pi
  * @rq: rq to get wqe from
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_qp.h b/drivers/net/ethernet/huawei/hinic/hinic_hw_qp.h
index 1331bd4..11bb708 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_qp.h
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_qp.h
@@ -16,6 +16,7 @@
 #ifndef HINIC_HW_QP_H
 #define HINIC_HW_QP_H
 
+#include <linux/kernel.h>
 #include <linux/types.h>
 #include <linux/sizes.h>
 #include <linux/pci.h>
@@ -27,6 +28,22 @@
 #include "hinic_hw_wq.h"
 #include "hinic_hw_qp_ctxt.h"
 
+#define HINIC_SQ_DB_INFO_PI_HI_SHIFT            0
+#define HINIC_SQ_DB_INFO_QID_SHIFT              8
+#define HINIC_SQ_DB_INFO_PATH_SHIFT             23
+#define HINIC_SQ_DB_INFO_COS_SHIFT              24
+#define HINIC_SQ_DB_INFO_TYPE_SHIFT             27
+
+#define HINIC_SQ_DB_INFO_PI_HI_MASK             0xFF
+#define HINIC_SQ_DB_INFO_QID_MASK               0x3FF
+#define HINIC_SQ_DB_INFO_PATH_MASK              0x1
+#define HINIC_SQ_DB_INFO_COS_MASK               0x7
+#define HINIC_SQ_DB_INFO_TYPE_MASK              0x1F
+
+#define HINIC_SQ_DB_INFO_SET(val, member)       \
+		(((u32)(val) & HINIC_SQ_DB_INFO_##member##_MASK) \
+		 << HINIC_SQ_DB_INFO_##member##_SHIFT)
+
 #define HINIC_SQ_WQEBB_SIZE                     64
 #define HINIC_RQ_WQEBB_SIZE                     32
 
@@ -38,6 +55,12 @@
 
 #define HINIC_RX_BUF_SZ                         2048
 
+#define HINIC_MIN_TX_WQE_SIZE(wq)               \
+		ALIGN(HINIC_SQ_WQE_SIZE(1), (wq)->wqebb_size)
+
+#define HINIC_MIN_TX_NUM_WQEBBS(sq)             \
+		(HINIC_MIN_TX_WQE_SIZE((sq)->wq) / (sq)->wq->wqebb_size)
+
 struct hinic_sq {
 	struct hinic_hwif       *hwif;
 
@@ -101,8 +124,32 @@ int hinic_init_rq(struct hinic_rq *rq, struct hinic_hwif *hwif,
 
 void hinic_clean_rq(struct hinic_rq *rq);
 
+int hinic_get_sq_free_wqebbs(struct hinic_sq *sq);
+
 int hinic_get_rq_free_wqebbs(struct hinic_rq *rq);
 
+void hinic_sq_prepare_wqe(struct hinic_sq *sq, u16 prod_idx,
+			  struct hinic_sq_wqe *wqe, struct hinic_sge *sges,
+			  int nr_sges);
+
+void hinic_sq_write_db(struct hinic_sq *sq, u16 prod_idx, unsigned int cos);
+
+struct hinic_sq_wqe *hinic_sq_get_wqe(struct hinic_sq *sq,
+				      unsigned int wqe_size, u16 *prod_idx);
+
+void hinic_sq_write_wqe(struct hinic_sq *sq, u16 prod_idx,
+			struct hinic_sq_wqe *wqe, struct sk_buff *skb,
+			unsigned int wqe_size);
+
+struct hinic_sq_wqe *hinic_sq_read_wqe(struct hinic_sq *sq,
+				       struct sk_buff **skb,
+				       unsigned int *wqe_size, u16 *cons_idx);
+
+void hinic_sq_put_wqe(struct hinic_sq *sq, unsigned int wqe_size);
+
+void hinic_sq_get_sges(struct hinic_sq_wqe *wqe, struct hinic_sge *sges,
+		       int nr_sges);
+
 struct hinic_rq_wqe *hinic_rq_get_wqe(struct hinic_rq *rq,
 				      unsigned int wqe_size, u16 *prod_idx);
 
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_main.c b/drivers/net/ethernet/huawei/hinic/hinic_main.c
index 4a99de1..3eb3fc1 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_main.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_main.c
@@ -43,6 +43,10 @@
 MODULE_DESCRIPTION("Huawei Intelligent NIC driver");
 MODULE_LICENSE("GPL");
 
+static unsigned int tx_weight = 64;
+module_param(tx_weight, uint, 0644);
+MODULE_PARM_DESC(tx_weight, "Number Tx packets for NAPI budget (default=64)");
+
 static unsigned int rx_weight = 64;
 module_param(rx_weight, uint, 0644);
 MODULE_PARM_DESC(rx_weight, "Number Rx packets for NAPI budget (default=64)");
@@ -577,9 +581,11 @@ static void hinic_set_rx_mode(struct net_device *netdev)
 	queue_work(nic_dev->workq, &rx_mode_work->work);
 }
 
-netdev_tx_t hinic_xmit_frame(struct sk_buff *skb, struct net_device *netdev)
+static void hinic_tx_timeout(struct net_device *netdev)
 {
-	return NETDEV_TX_BUSY;
+	struct hinic_dev *nic_dev = netdev_priv(netdev);
+
+	netif_err(nic_dev, drv, netdev, "Tx timeout\n");
 }
 
 static const struct net_device_ops hinic_netdev_ops = {
@@ -592,6 +598,7 @@ netdev_tx_t hinic_xmit_frame(struct sk_buff *skb, struct net_device *netdev)
 	.ndo_vlan_rx_kill_vid = hinic_vlan_rx_kill_vid,
 	.ndo_set_rx_mode = hinic_set_rx_mode,
 	.ndo_start_xmit = hinic_xmit_frame,
+	.ndo_tx_timeout = hinic_tx_timeout,
 	/* more operations should be filled */
 };
 
@@ -699,6 +706,7 @@ static int nic_dev_init(struct pci_dev *pdev)
 	nic_dev->flags = 0;
 	nic_dev->txqs = NULL;
 	nic_dev->rxqs = NULL;
+	nic_dev->tx_weight = tx_weight;
 	nic_dev->rx_weight = rx_weight;
 
 	sema_init(&nic_dev->mgmt_lock, 1);
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_tx.c b/drivers/net/ethernet/huawei/hinic/hinic_tx.c
index 9c27fb2..c3e36ab 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_tx.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_tx.c
@@ -13,12 +13,42 @@
  *
  */
 
+#include <linux/kernel.h>
 #include <linux/netdevice.h>
 #include <linux/u64_stats_sync.h>
+#include <linux/errno.h>
+#include <linux/types.h>
+#include <linux/pci.h>
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/slab.h>
+#include <linux/interrupt.h>
+#include <linux/skbuff.h>
+#include <linux/smp.h>
+#include <asm/byteorder.h>
 
+#include "hinic_common.h"
+#include "hinic_hw_if.h"
+#include "hinic_hw_wqe.h"
+#include "hinic_hw_wq.h"
 #include "hinic_hw_qp.h"
+#include "hinic_hw_dev.h"
+#include "hinic_dev.h"
 #include "hinic_tx.h"
 
+#define TX_IRQ_NO_PENDING               0
+#define TX_IRQ_NO_COALESC               0
+#define TX_IRQ_NO_LLI_TIMER             0
+#define TX_IRQ_NO_CREDIT                0
+#define TX_IRQ_NO_RESEND_TIMER          0
+
+#define CI_UPDATE_NO_PENDING            0
+#define CI_UPDATE_NO_COALESC            0
+
+#define HW_CONS_IDX(sq)         be16_to_cpu(*(u16 *)((sq)->hw_ci_addr))
+
+#define MIN_SKB_LEN             64
+
 /**
  * hinic_txq_clean_stats - Clean the statistics of specific queue
  * @txq: Logical Tx Queue
@@ -49,6 +79,329 @@ static void txq_stats_init(struct hinic_txq *txq)
 }
 
 /**
+ * tx_map_skb - dma mapping for skb and return sges
+ * @nic_dev: nic device
+ * @skb: the skb
+ * @sges: returned sges
+ *
+ * Return 0 - Success, negative - Failure
+ **/
+static int tx_map_skb(struct hinic_dev *nic_dev, struct sk_buff *skb,
+		      struct hinic_sge *sges)
+{
+	struct hinic_hwdev *hwdev = nic_dev->hwdev;
+	struct hinic_hwif *hwif = hwdev->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+	dma_addr_t dma_addr;
+	struct skb_frag_struct *frag;
+	int i, j;
+
+	dma_addr = dma_map_single(&pdev->dev, skb->data, skb_headlen(skb),
+				  DMA_TO_DEVICE);
+	if (dma_mapping_error(&pdev->dev, dma_addr)) {
+		dev_err(&pdev->dev, "Failed to map Tx skb data\n");
+		return -EFAULT;
+	}
+
+	hinic_set_sge(&sges[0], dma_addr, skb_headlen(skb));
+
+	for (i = 0 ; i < skb_shinfo(skb)->nr_frags; i++) {
+		frag = &(skb_shinfo(skb)->frags[i]);
+
+		dma_addr = skb_frag_dma_map(&pdev->dev, frag, 0,
+					    skb_frag_size(frag),
+					    DMA_TO_DEVICE);
+		if (dma_mapping_error(&pdev->dev, dma_addr)) {
+			dev_err(&pdev->dev, "Failed to map Tx skb frag\n");
+			goto err_tx_map;
+		}
+
+		hinic_set_sge(&sges[i + 1], dma_addr, skb_frag_size(frag));
+	}
+
+	return 0;
+
+err_tx_map:
+	for (j = 0; j < i; j++)
+		dma_unmap_page(&pdev->dev, hinic_sge_to_dma(&sges[j + 1]),
+			       sges[j + 1].len, DMA_TO_DEVICE);
+
+	dma_unmap_single(&pdev->dev, hinic_sge_to_dma(&sges[0]), sges[0].len,
+			 DMA_TO_DEVICE);
+	return -EFAULT;
+}
+
+/**
+ * tx_unmap_skb - unmap the dma address of the skb
+ * @nic_dev: nic device
+ * @skb: the skb
+ * @sges: the sges that are connected to the skb
+ **/
+static void tx_unmap_skb(struct hinic_dev *nic_dev, struct sk_buff *skb,
+			 struct hinic_sge *sges)
+{
+	struct hinic_hwdev *hwdev = nic_dev->hwdev;
+	struct hinic_hwif *hwif = hwdev->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+	int i;
+
+	for (i = 0; i < skb_shinfo(skb)->nr_frags ; i++)
+		dma_unmap_page(&pdev->dev, hinic_sge_to_dma(&sges[i + 1]),
+			       sges[i + 1].len, DMA_TO_DEVICE);
+
+	dma_unmap_single(&pdev->dev, hinic_sge_to_dma(&sges[0]), sges[0].len,
+			 DMA_TO_DEVICE);
+}
+
+netdev_tx_t hinic_xmit_frame(struct sk_buff *skb, struct net_device *netdev)
+{
+	struct hinic_dev *nic_dev = netdev_priv(netdev);
+	struct hinic_hwdev *hwdev = nic_dev->hwdev;
+	struct hinic_hwif *hwif = hwdev->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+	int qpn = skb->queue_mapping;
+	int err = NETDEV_TX_OK, cos = 0;
+	struct netdev_queue *netdev_txq = netdev_get_tx_queue(netdev, qpn);
+	struct hinic_txq *txq = &nic_dev->txqs[qpn];
+	struct hinic_sq *sq = txq->sq;
+	struct hinic_wq *wq = sq->wq;
+	struct hinic_qp *qp = container_of(sq, struct hinic_qp, sq);
+	struct hinic_sge *sges = txq->sges;
+	struct hinic_sq_wqe *sq_wqe;
+	unsigned int wqe_size;
+	u16 prod_idx;
+	int nr_sges;
+
+	if (skb->len < MIN_SKB_LEN) {
+		if (skb_pad(skb, MIN_SKB_LEN - skb->len)) {
+			dev_err(&netdev->dev, "Failed to pad skb\n");
+			goto skb_error;
+		}
+
+		skb->len = MIN_SKB_LEN;
+	}
+
+	nr_sges = skb_shinfo(skb)->nr_frags + 1;
+	if (nr_sges > txq->max_sges) {
+		dev_err(&pdev->dev, "Too many Tx sges\n");
+		goto skb_error;
+	}
+
+	err = tx_map_skb(nic_dev, skb, sges);
+	if (err) {
+		dev_err(&pdev->dev, "Failed to map Tx sges\n");
+		goto skb_error;
+	}
+
+	wqe_size = HINIC_SQ_WQE_SIZE(nr_sges);
+
+	sq_wqe = hinic_sq_get_wqe(txq->sq, wqe_size, &prod_idx);
+	if (!sq_wqe) {
+		dev_err(&pdev->dev, "Failed to get SQ WQE\n");
+		tx_unmap_skb(nic_dev, skb, sges);
+
+		netif_stop_subqueue(netdev, qp->q_id);
+
+		u64_stats_update_begin(&txq->txq_stats.syncp);
+		txq->txq_stats.tx_busy++;
+		u64_stats_update_end(&txq->txq_stats.syncp);
+		err = NETDEV_TX_BUSY;
+		goto flush_skbs;
+	}
+
+	hinic_sq_prepare_wqe(txq->sq, prod_idx, sq_wqe, sges, nr_sges);
+
+	hinic_sq_write_wqe(txq->sq, prod_idx, sq_wqe, skb, wqe_size);
+
+	/* increment prod_idx to the next */
+	prod_idx += ALIGN(wqe_size, wq->wqebb_size) / wq->wqebb_size;
+
+flush_skbs:
+	if ((!skb->xmit_more) || (netif_xmit_stopped(netdev_txq)))
+		hinic_sq_write_db(txq->sq, prod_idx, cos);
+
+	return err;
+
+skb_error:
+	dev_kfree_skb_any(skb);
+
+	u64_stats_update_begin(&txq->txq_stats.syncp);
+	txq->txq_stats.tx_dropped++;
+	u64_stats_update_end(&txq->txq_stats.syncp);
+	return err;
+}
+
+/**
+ * tx_free_skb - unmap and free skb
+ * @nic_dev: nic device
+ * @skb: the skb
+ * @sges: the sges that are connected to the skb
+ **/
+static void tx_free_skb(struct hinic_dev *nic_dev, struct sk_buff *skb,
+			struct hinic_sge *sges)
+{
+	tx_unmap_skb(nic_dev, skb, sges);
+
+	dev_kfree_skb_any(skb);
+}
+
+/**
+ * free_all_rx_skbs - free all skbs in tx queue
+ * @txq: tx queue
+ **/
+static void free_all_tx_skbs(struct hinic_txq *txq)
+{
+	struct hinic_dev *nic_dev = netdev_priv(txq->netdev);
+	struct hinic_sq *sq = txq->sq;
+	struct hinic_sq_wqe *sq_wqe;
+	struct sk_buff *skb;
+	u16 ci;
+	int wqe_size, nr_sges;
+
+	while ((sq_wqe = hinic_sq_read_wqe(sq, &skb, &wqe_size, &ci))) {
+		nr_sges = skb_shinfo(skb)->nr_frags + 1;
+
+		hinic_sq_get_sges(sq_wqe, txq->free_sges, nr_sges);
+
+		hinic_sq_put_wqe(sq, wqe_size);
+
+		tx_free_skb(nic_dev, skb, txq->free_sges);
+	}
+}
+
+/**
+ * free_tx_poll - free finished tx skbs in tx queue that connected to napi
+ * @napi: napi
+ * @budget: number of tx
+ *
+ * Return 0 - Success, negative - Failure
+ **/
+static int free_tx_poll(struct napi_struct *napi, int budget)
+{
+	struct hinic_txq *txq = container_of(napi, struct hinic_txq, napi);
+	struct hinic_dev *nic_dev = netdev_priv(txq->netdev);
+	struct hinic_sq *sq = txq->sq;
+	struct hinic_qp *qp = container_of(sq, struct hinic_qp, sq);
+	struct hinic_wq *wq = sq->wq;
+	struct netdev_queue *netdev_txq;
+	struct hinic_sq_wqe *sq_wqe;
+	struct sk_buff *skb;
+	u64 tx_bytes = 0;
+	unsigned int wqe_size;
+	int nr_sges, pkts = 0;
+	u16 hw_ci, sw_ci, q_id = qp->q_id;
+
+	do {
+		hw_ci = HW_CONS_IDX(sq) & wq->mask;
+
+		sq_wqe = hinic_sq_read_wqe(sq, &skb, &wqe_size, &sw_ci);
+		if ((!sq_wqe) ||
+		    (((hw_ci - sw_ci) & wq->mask) * wq->wqebb_size < wqe_size))
+			break;
+
+		tx_bytes += skb->len;
+		pkts++;
+
+		nr_sges = skb_shinfo(skb)->nr_frags + 1;
+
+		hinic_sq_get_sges(sq_wqe, txq->free_sges, nr_sges);
+
+		hinic_sq_put_wqe(sq, wqe_size);
+
+		tx_free_skb(nic_dev, skb, txq->free_sges);
+	} while (pkts < budget);
+
+	if (__netif_subqueue_stopped(nic_dev->netdev, q_id) &&
+	    hinic_get_sq_free_wqebbs(sq) >= HINIC_MIN_TX_NUM_WQEBBS(sq)) {
+		netdev_txq = netdev_get_tx_queue(txq->netdev, q_id);
+
+		__netif_tx_lock(netdev_txq, smp_processor_id());
+
+		netif_wake_subqueue(nic_dev->netdev, q_id);
+
+		__netif_tx_unlock(netdev_txq);
+
+		u64_stats_update_begin(&txq->txq_stats.syncp);
+		txq->txq_stats.tx_wake++;
+		u64_stats_update_end(&txq->txq_stats.syncp);
+	}
+
+	u64_stats_update_begin(&txq->txq_stats.syncp);
+	txq->txq_stats.bytes += tx_bytes;
+	txq->txq_stats.pkts += pkts;
+	u64_stats_update_end(&txq->txq_stats.syncp);
+
+	if (pkts < budget) {
+		napi_complete(napi);
+		enable_irq(sq->irq);
+		return pkts;
+	}
+
+	return budget;
+}
+
+static void tx_napi_add(struct hinic_txq *txq, int weight)
+{
+	netif_napi_add(txq->netdev, &txq->napi, free_tx_poll, weight);
+	napi_enable(&txq->napi);
+}
+
+static void tx_napi_del(struct hinic_txq *txq)
+{
+	napi_disable(&txq->napi);
+	netif_napi_del(&txq->napi);
+}
+
+static irqreturn_t tx_irq(int irq, void *data)
+{
+	struct hinic_txq *txq = (struct hinic_txq *)data;
+	struct hinic_dev *nic_dev = netdev_priv(txq->netdev);
+	struct hinic_sq *sq = txq->sq;
+
+	/* Disable the interrupt until napi will be completed */
+	disable_irq_nosync(sq->irq);
+
+	hinic_hwdev_msix_cnt_set(nic_dev->hwdev, sq->msix_entry);
+
+	napi_schedule(&txq->napi);
+	return IRQ_HANDLED;
+}
+
+static int tx_request_irq(struct hinic_txq *txq)
+{
+	struct hinic_dev *nic_dev = netdev_priv(txq->netdev);
+	struct hinic_hwdev *hwdev = nic_dev->hwdev;
+	struct hinic_hwif *hwif = hwdev->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+	struct hinic_sq *sq = txq->sq;
+	int err;
+
+	tx_napi_add(txq, nic_dev->tx_weight);
+
+	hinic_hwdev_msix_set(nic_dev->hwdev, sq->msix_entry,
+			     TX_IRQ_NO_PENDING, TX_IRQ_NO_COALESC,
+			     TX_IRQ_NO_LLI_TIMER, TX_IRQ_NO_CREDIT,
+			     TX_IRQ_NO_RESEND_TIMER);
+
+	err = request_irq(sq->irq, tx_irq, 0, txq->irq_name, txq);
+	if (err) {
+		dev_err(&pdev->dev, "Failed to request Tx irq\n");
+		tx_napi_del(txq);
+		return err;
+	}
+
+	return 0;
+}
+
+static void tx_free_irq(struct hinic_txq *txq)
+{
+	struct hinic_sq *sq = txq->sq;
+
+	free_irq(sq->irq, txq);
+	tx_napi_del(txq);
+}
+
+/**
  * hinic_init_txq - Initialize the Tx Queue
  * @txq: Logical Tx Queue
  * @sq: Hardware Tx Queue to connect the Logical queue with
@@ -59,11 +412,68 @@ static void txq_stats_init(struct hinic_txq *txq)
 int hinic_init_txq(struct hinic_txq *txq, struct hinic_sq *sq,
 		   struct net_device *netdev)
 {
+	struct hinic_dev *nic_dev = netdev_priv(netdev);
+	struct hinic_hwdev *hwdev = nic_dev->hwdev;
+	struct hinic_hwif *hwif = hwdev->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+	struct hinic_qp *qp = container_of(sq, struct hinic_qp, sq);
+	size_t sges_size;
+	int err, irqname_len;
+
 	txq->netdev = netdev;
 	txq->sq = sq;
 
 	txq_stats_init(txq);
+
+	txq->max_sges = HINIC_MAX_SQ_BUFDESCS;
+
+	sges_size = txq->max_sges * sizeof(*txq->sges);
+	txq->sges = devm_kzalloc(&netdev->dev, sges_size, GFP_KERNEL);
+	if (!txq->sges)
+		return -ENOMEM;
+
+	sges_size = txq->max_sges * sizeof(*txq->free_sges);
+	txq->free_sges = devm_kzalloc(&netdev->dev, sges_size, GFP_KERNEL);
+	if (!txq->free_sges) {
+		err = -ENOMEM;
+		goto err_alloc_free_sges;
+	}
+
+	irqname_len = snprintf(NULL, 0, "hinic_txq%d", qp->q_id) + 1;
+	txq->irq_name = devm_kzalloc(&netdev->dev, irqname_len, GFP_KERNEL);
+	if (!txq->irq_name) {
+		err = -ENOMEM;
+		goto err_alloc_irqname;
+	}
+
+	sprintf(txq->irq_name, "hinic_txq%d", qp->q_id);
+
+	err = hinic_hwdev_hw_ci_addr_set(hwdev, sq, CI_UPDATE_NO_PENDING,
+					 CI_UPDATE_NO_COALESC);
+	if (err) {
+		dev_err(&pdev->dev, "Failed to set HW CI for qid = %d\n",
+			qp->q_id);
+		goto err_hw_ci;
+	}
+
+	err = tx_request_irq(txq);
+	if (err) {
+		dev_err(&pdev->dev, "Failed to request Tx irq\n");
+		goto err_req_tx_irq;
+	}
+
 	return 0;
+
+err_req_tx_irq:
+err_hw_ci:
+	devm_kfree(&netdev->dev, txq->irq_name);
+
+err_alloc_irqname:
+	devm_kfree(&netdev->dev, txq->free_sges);
+
+err_alloc_free_sges:
+	devm_kfree(&netdev->dev, txq->sges);
+	return err;
 }
 
 /**
@@ -72,4 +482,13 @@ int hinic_init_txq(struct hinic_txq *txq, struct hinic_sq *sq,
  **/
 void hinic_clean_txq(struct hinic_txq *txq)
 {
+	struct net_device *netdev = txq->netdev;
+
+	tx_free_irq(txq);
+
+	free_all_tx_skbs(txq);
+
+	devm_kfree(&netdev->dev, txq->irq_name);
+	devm_kfree(&netdev->dev, txq->free_sges);
+	devm_kfree(&netdev->dev, txq->sges);
 }
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_tx.h b/drivers/net/ethernet/huawei/hinic/hinic_tx.h
index bbdb4b6..7123c7f 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_tx.h
+++ b/drivers/net/ethernet/huawei/hinic/hinic_tx.h
@@ -18,8 +18,10 @@
 
 #include <linux/types.h>
 #include <linux/netdevice.h>
+#include <linux/skbuff.h>
 #include <linux/u64_stats_sync.h>
 
+#include "hinic_common.h"
 #include "hinic_hw_qp.h"
 
 struct hinic_txq_stats {
@@ -37,10 +39,19 @@ struct hinic_txq {
 	struct hinic_sq         *sq;
 
 	struct hinic_txq_stats  txq_stats;
+
+	int                     max_sges;
+	struct hinic_sge        *sges;
+	struct hinic_sge        *free_sges;
+
+	char                    *irq_name;
+	struct napi_struct      napi;
 };
 
 void hinic_txq_clean_stats(struct hinic_txq *txq);
 
+netdev_tx_t hinic_xmit_frame(struct sk_buff *skb, struct net_device *netdev);
+
 int hinic_init_txq(struct hinic_txq *txq, struct hinic_sq *sq,
 		   struct net_device *netdev);
 
-- 
1.9.1

[toc] | [prev] | [next] | [standalone]


#1702897 — [PATCH V3 net-next 21/21] net-next/hinic: Add select_queue and netpoll

FromAviad Krawczyk <aviad.krawczyk@huawei.com>
Date2017-08-03 12:10 +0200
Subject[PATCH V3 net-next 21/21] net-next/hinic: Add select_queue and netpoll
Message-ID<ualeR-6Uy-67@gated-at.bofh.it>
In reply to#1702870
Add more netdev operations.

Signed-off-by: Aviad Krawczyk <aviad.krawczyk@huawei.com>
Signed-off-by: Zhao Chen <zhaochen6@huawei.com>
---
 MAINTAINERS                                    |  7 ++++++
 drivers/net/ethernet/huawei/hinic/hinic_main.c | 35 ++++++++++++++++++++++++++
 2 files changed, 42 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 207e453..da06ef9 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -6232,6 +6232,13 @@ L:	linux-input@vger.kernel.org
 S:	Maintained
 F:	drivers/input/touchscreen/htcpen.c
 
+HUAWEI ETHERNET DRIVER
+M:	Aviad Krawczyk <aviad.krawczyk@huawei.com>
+L:	netdev@vger.kernel.org
+S:	Supported
+F:	Documentation/networking/hinic.txt
+F:	drivers/net/ethernet/huawei/hinic/
+
 HUGETLB FILESYSTEM
 M:	Nadia Yvette Chambers <nyc@holomorphy.com>
 S:	Maintained
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_main.c b/drivers/net/ethernet/huawei/hinic/hinic_main.c
index 4e0709a..c3426f8 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_main.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_main.c
@@ -797,6 +797,37 @@ static void hinic_get_stats64(struct net_device *netdev,
 	stats->tx_errors = nic_tx_stats->tx_dropped;
 }
 
+static u16 hinic_select_queue(struct net_device *netdev, struct sk_buff *skb,
+			      void *accel_priv,
+			      select_queue_fallback_t fallback)
+{
+	u16 qid;
+
+	if (skb_rx_queue_recorded(skb))
+		qid = skb_get_rx_queue(skb);
+	else
+		qid = fallback(netdev, skb);
+
+	return qid;
+}
+
+#ifdef CONFIG_NET_POLL_CONTROLLER
+static void hinic_netpoll(struct net_device *netdev)
+{
+	struct hinic_dev *nic_dev = netdev_priv(netdev);
+	struct hinic_hwdev *hwdev = nic_dev->hwdev;
+	int i, num_qps = hinic_hwdev_num_qps(hwdev);
+
+	for (i = 0; i < num_qps; i++) {
+		struct hinic_txq *txq = &nic_dev->txqs[i];
+		struct hinic_rxq *rxq = &nic_dev->rxqs[i];
+
+		napi_schedule(&txq->napi);
+		napi_schedule(&rxq->napi);
+	}
+}
+#endif
+
 static const struct net_device_ops hinic_netdev_ops = {
 	.ndo_open = hinic_open,
 	.ndo_stop = hinic_close,
@@ -809,6 +840,10 @@ static void hinic_get_stats64(struct net_device *netdev,
 	.ndo_start_xmit = hinic_xmit_frame,
 	.ndo_tx_timeout = hinic_tx_timeout,
 	.ndo_get_stats64 = hinic_get_stats64,
+	.ndo_select_queue = hinic_select_queue,
+#ifdef CONFIG_NET_POLL_CONTROLLER
+	.ndo_poll_controller = hinic_netpoll,
+#endif
 };
 
 static void netdev_features_init(struct net_device *netdev)
-- 
1.9.1

[toc] | [prev] | [next] | [standalone]


#1702901 — [PATCH V3 net-next 06/21] net-next/hinic: Add api cmd commands

FromAviad Krawczyk <aviad.krawczyk@huawei.com>
Date2017-08-03 12:20 +0200
Subject[PATCH V3 net-next 06/21] net-next/hinic: Add api cmd commands
Message-ID<ualot-6Yh-13@gated-at.bofh.it>
In reply to#1702870
Add the api cmd commands for sending management messages to the nic.

Signed-off-by: Aviad Krawczyk <aviad.krawczyk@huawei.com>
Signed-off-by: Zhao Chen <zhaochen6@huawei.com>
---
 .../net/ethernet/huawei/hinic/hinic_hw_api_cmd.c   | 329 ++++++++++++++++++++-
 .../net/ethernet/huawei/hinic/hinic_hw_api_cmd.h   |  65 ++++
 drivers/net/ethernet/huawei/hinic/hinic_hw_csr.h   |   7 +
 3 files changed, 399 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.c b/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.c
index a99d87c..04172fe 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.c
@@ -25,7 +25,9 @@
 #include <linux/jiffies.h>
 #include <linux/delay.h>
 #include <linux/log2.h>
+#include <linux/semaphore.h>
 #include <asm/byteorder.h>
+#include <asm/barrier.h>
 
 #include "hinic_hw_csr.h"
 #include "hinic_hw_if.h"
@@ -45,14 +47,313 @@
 
 #define API_CMD_BUF_SIZE                        2048
 
+/* Sizes of the members in hinic_api_cmd_cell */
+#define API_CMD_CELL_DESC_SIZE          8
+#define API_CMD_CELL_DATA_ADDR_SIZE     8
+
+#define API_CMD_CELL_ALIGNMENT          8
+
 #define API_CMD_TIMEOUT                 1000
 
+#define MASKED_IDX(chain, idx)          ((idx) & ((chain)->num_cells - 1))
+
+#define SIZE_8BYTES(size)               (ALIGN((size), 8) >> 3)
+#define SIZE_4BYTES(size)               (ALIGN((size), 4) >> 2)
+
+#define RD_DMA_ATTR_DEFAULT             0
+#define WR_DMA_ATTR_DEFAULT             0
+
+enum api_cmd_data_format {
+	SGE_DATA = 1,           /* cell data is passed by hw address */
+};
+
+enum api_cmd_type {
+	API_CMD_WRITE = 0,
+};
+
+enum api_cmd_bypass {
+	NO_BYPASS       = 0,
+	BYPASS          = 1,
+};
+
 enum api_cmd_xor_chk_level {
 	XOR_CHK_DIS = 0,
 
 	XOR_CHK_ALL = 3,
 };
 
+static u8 xor_chksum_set(void *data)
+{
+	int idx;
+	u8 *val, checksum = 0;
+
+	val = data;
+
+	for (idx = 0; idx < 7; idx++)
+		checksum ^= val[idx];
+
+	return checksum;
+}
+
+static void set_prod_idx(struct hinic_api_cmd_chain *chain)
+{
+	struct hinic_hwif *hwif = chain->hwif;
+	enum hinic_api_cmd_chain_type chain_type = chain->chain_type;
+	u32 hw_prod_idx_addr = HINIC_CSR_API_CMD_CHAIN_PI_ADDR(chain_type);
+	u32 prod_idx;
+
+	prod_idx = hinic_hwif_read_reg(hwif, hw_prod_idx_addr);
+
+	prod_idx = HINIC_API_CMD_PI_CLEAR(prod_idx, IDX);
+
+	prod_idx |= HINIC_API_CMD_PI_SET(chain->prod_idx, IDX);
+
+	hinic_hwif_write_reg(hwif, hw_prod_idx_addr, prod_idx);
+}
+
+static u32 get_hw_cons_idx(struct hinic_api_cmd_chain *chain)
+{
+	u32 addr, val;
+
+	addr = HINIC_CSR_API_CMD_STATUS_ADDR(chain->chain_type);
+	val  = hinic_hwif_read_reg(chain->hwif, addr);
+
+	return HINIC_API_CMD_STATUS_GET(val, CONS_IDX);
+}
+
+/**
+ * chain_busy - check if the chain is still processing last requests
+ * @chain: chain to check
+ *
+ * Return 0 - Success, negative - Failure
+ **/
+static int chain_busy(struct hinic_api_cmd_chain *chain)
+{
+	struct hinic_hwif *hwif = chain->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+	u32 prod_idx;
+
+	switch (chain->chain_type) {
+	case HINIC_API_CMD_WRITE_TO_MGMT_CPU:
+		chain->cons_idx = get_hw_cons_idx(chain);
+		prod_idx = chain->prod_idx;
+
+		/* check for a space for a new command */
+		if (chain->cons_idx == MASKED_IDX(chain, prod_idx + 1)) {
+			dev_err(&pdev->dev, "API CMD chain %d is busy\n",
+				chain->chain_type);
+			return -EBUSY;
+		}
+		break;
+
+	default:
+		dev_err(&pdev->dev, "Unknown API CMD Chain type\n");
+		break;
+	}
+
+	return 0;
+}
+
+/**
+ * get_cell_data_size - get the data size of a specific cell type
+ * @type: chain type
+ *
+ * Return the data(Desc + Address) size in the cell
+ **/
+static u8 get_cell_data_size(enum hinic_api_cmd_chain_type type)
+{
+	u8 cell_data_size = 0;
+
+	switch (type) {
+	case HINIC_API_CMD_WRITE_TO_MGMT_CPU:
+		cell_data_size = ALIGN(API_CMD_CELL_DESC_SIZE +
+				       API_CMD_CELL_DATA_ADDR_SIZE,
+				       API_CMD_CELL_ALIGNMENT);
+		break;
+	default:
+		break;
+	}
+
+	return cell_data_size;
+}
+
+/**
+ * prepare_cell_ctrl - prepare the ctrl of the cell for the command
+ * @cell_ctrl: the control of the cell to set the control value into it
+ * @data_size: the size of the data in the cell
+ **/
+static void prepare_cell_ctrl(u64 *cell_ctrl, u16 data_size)
+{
+	u64 ctrl;
+	u8 chksum;
+
+	ctrl =  HINIC_API_CMD_CELL_CTRL_SET(SIZE_8BYTES(data_size), DATA_SZ) |
+		HINIC_API_CMD_CELL_CTRL_SET(RD_DMA_ATTR_DEFAULT, RD_DMA_ATTR) |
+		HINIC_API_CMD_CELL_CTRL_SET(WR_DMA_ATTR_DEFAULT, WR_DMA_ATTR);
+
+	chksum = xor_chksum_set(&ctrl);
+
+	ctrl |= HINIC_API_CMD_CELL_CTRL_SET(chksum, XOR_CHKSUM);
+
+	/* The data in the HW should be in Big Endian Format */
+	*cell_ctrl = cpu_to_be64(ctrl);
+}
+
+/**
+ * prepare_api_cmd - prepare API CMD command
+ * @chain: chain for the command
+ * @dest: destination node on the card that will receive the command
+ * @cmd: command data
+ * @cmd_size: the command size
+ **/
+static void prepare_api_cmd(struct hinic_api_cmd_chain *chain,
+			    enum hinic_node_id dest,
+			    void *cmd, u16 cmd_size)
+{
+	struct hinic_api_cmd_cell *cell = chain->curr_node;
+	struct hinic_hwif *hwif = chain->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+	struct hinic_api_cmd_cell_ctxt *cell_ctxt;
+
+	cell_ctxt = &chain->cell_ctxt[chain->prod_idx];
+
+	switch (chain->chain_type) {
+	case HINIC_API_CMD_WRITE_TO_MGMT_CPU:
+		cell->desc = HINIC_API_CMD_DESC_SET(SGE_DATA, API_TYPE) |
+			     HINIC_API_CMD_DESC_SET(API_CMD_WRITE, RD_WR) |
+			     HINIC_API_CMD_DESC_SET(NO_BYPASS, MGMT_BYPASS);
+		break;
+
+	default:
+		dev_err(&pdev->dev, "unknown Chain type\n");
+		return;
+	}
+
+	cell->desc |= HINIC_API_CMD_DESC_SET(dest, DEST)        |
+		      HINIC_API_CMD_DESC_SET(SIZE_4BYTES(cmd_size), SIZE);
+
+	cell->desc |= HINIC_API_CMD_DESC_SET(xor_chksum_set(&cell->desc),
+					     XOR_CHKSUM);
+
+	/* The data in the HW should be in Big Endian Format */
+	cell->desc = cpu_to_be64(cell->desc);
+
+	memcpy(cell_ctxt->api_cmd_vaddr, cmd, cmd_size);
+}
+
+/**
+ * prepare_cell - prepare cell ctrl and cmd in the current cell
+ * @chain: chain for the command
+ * @dest: destination node on the card that will receive the command
+ * @cmd: command data
+ * @cmd_size: the command size
+ *
+ * Return 0 - Success, negative - Failure
+ **/
+static void prepare_cell(struct hinic_api_cmd_chain *chain,
+			 enum  hinic_node_id dest,
+			 void *cmd, u16 cmd_size)
+{
+	struct hinic_api_cmd_cell *curr_node = chain->curr_node;
+	u16 data_size = get_cell_data_size(chain->chain_type);
+
+	prepare_cell_ctrl(&curr_node->ctrl, data_size);
+	prepare_api_cmd(chain, dest, cmd, cmd_size);
+}
+
+static inline void cmd_chain_prod_idx_inc(struct hinic_api_cmd_chain *chain)
+{
+	chain->prod_idx = MASKED_IDX(chain, chain->prod_idx + 1);
+}
+
+/**
+ * api_cmd_status_update - update the status in the chain struct
+ * @chain: chain to update
+ **/
+static void api_cmd_status_update(struct hinic_api_cmd_chain *chain)
+{
+	struct hinic_hwif *hwif = chain->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+	struct hinic_api_cmd_status *wb_status;
+	enum hinic_api_cmd_chain_type chain_type;
+	u64     status_header;
+	u32     status;
+
+	wb_status = chain->wb_status;
+	status_header = be64_to_cpu(wb_status->header);
+
+	status = be32_to_cpu(wb_status->status);
+	if (HINIC_API_CMD_STATUS_GET(status, CHKSUM_ERR)) {
+		dev_err(&pdev->dev, "API CMD status: Xor check error\n");
+		return;
+	}
+
+	chain_type = HINIC_API_CMD_STATUS_HEADER_GET(status_header, CHAIN_ID);
+	if (chain_type >= HINIC_API_CMD_MAX) {
+		dev_err(&pdev->dev, "unknown API CMD Chain %d\n", chain_type);
+		return;
+	}
+
+	chain->cons_idx = HINIC_API_CMD_STATUS_GET(status, CONS_IDX);
+}
+
+/**
+ * wait_for_status_poll - wait for write to api cmd command to complete
+ * @chain: the chain of the command
+ *
+ * Return 0 - Success, negative - Failure
+ **/
+static int wait_for_status_poll(struct hinic_api_cmd_chain *chain)
+{
+	unsigned long end;
+	int err = -ETIMEDOUT;
+
+	end = jiffies + msecs_to_jiffies(API_CMD_TIMEOUT);
+	do {
+		api_cmd_status_update(chain);
+
+		/* wait for CI to be updated - sign for completion */
+		if (chain->cons_idx == chain->prod_idx) {
+			err = 0;
+			break;
+		}
+
+		msleep(20);
+	} while (time_before(jiffies, end));
+
+	return err;
+}
+
+/**
+ * wait_for_api_cmd_completion - wait for command to complete
+ * @chain: chain for the command
+ *
+ * Return 0 - Success, negative - Failure
+ **/
+static int wait_for_api_cmd_completion(struct hinic_api_cmd_chain *chain)
+{
+	struct hinic_hwif *hwif = chain->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+	int err;
+
+	switch (chain->chain_type) {
+	case HINIC_API_CMD_WRITE_TO_MGMT_CPU:
+		err = wait_for_status_poll(chain);
+		if (err) {
+			dev_err(&pdev->dev, "API CMD Poll status timeout\n");
+			break;
+		}
+		break;
+
+	default:
+		dev_err(&pdev->dev, "unknown API CMD Chain type\n");
+		err = -EINVAL;
+		break;
+	}
+
+	return err;
+}
+
 /**
  * api_cmd - API CMD command
  * @chain: chain for the command
@@ -65,8 +366,30 @@ enum api_cmd_xor_chk_level {
 static int api_cmd(struct hinic_api_cmd_chain *chain,
 		   enum hinic_node_id dest, u8 *cmd, u16 cmd_size)
 {
-	/* should be implemented */
-	return -EINVAL;
+	struct hinic_api_cmd_cell_ctxt *ctxt;
+	int err;
+
+	down(&chain->sem);
+	if (chain_busy(chain)) {
+		up(&chain->sem);
+		return -EBUSY;
+	}
+
+	prepare_cell(chain, dest, cmd, cmd_size);
+	cmd_chain_prod_idx_inc(chain);
+
+	wmb();  /* inc pi before issue the command */
+
+	set_prod_idx(chain);    /* issue the command */
+
+	ctxt = &chain->cell_ctxt[chain->prod_idx];
+
+	chain->curr_node = ctxt->cell_vaddr;
+
+	err = wait_for_api_cmd_completion(chain);
+
+	up(&chain->sem);
+	return err;
 }
 
 /**
@@ -491,6 +814,8 @@ static int api_chain_init(struct hinic_api_cmd_chain *chain,
 	chain->prod_idx  = 0;
 	chain->cons_idx  = 0;
 
+	sema_init(&chain->sem, 1);
+
 	cell_ctxt_size = chain->num_cells * sizeof(*chain->cell_ctxt);
 	chain->cell_ctxt = devm_kzalloc(&pdev->dev, cell_ctxt_size, GFP_KERNEL);
 	if (!chain->cell_ctxt)
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.h b/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.h
index e8865d6..31b94d5 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.h
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.h
@@ -17,9 +17,22 @@
 #define HINIC_HW_API_CMD_H
 
 #include <linux/types.h>
+#include <linux/semaphore.h>
 
 #include "hinic_hw_if.h"
 
+#define HINIC_API_CMD_PI_IDX_SHIFT                              0
+
+#define HINIC_API_CMD_PI_IDX_MASK                               0xFFFFFF
+
+#define HINIC_API_CMD_PI_SET(val, member)                       \
+	(((u32)(val) & HINIC_API_CMD_PI_##member##_MASK) <<     \
+	 HINIC_API_CMD_PI_##member##_SHIFT)
+
+#define HINIC_API_CMD_PI_CLEAR(val, member)                     \
+	((val) & (~(HINIC_API_CMD_PI_##member##_MASK            \
+	 << HINIC_API_CMD_PI_##member##_SHIFT)))
+
 #define HINIC_API_CMD_CHAIN_REQ_RESTART_SHIFT                   1
 
 #define HINIC_API_CMD_CHAIN_REQ_RESTART_MASK                    0x1
@@ -58,6 +71,56 @@
 	((val) & (~(HINIC_API_CMD_CHAIN_CTRL_##member##_MASK    \
 	 << HINIC_API_CMD_CHAIN_CTRL_##member##_SHIFT)))
 
+#define HINIC_API_CMD_CELL_CTRL_DATA_SZ_SHIFT                   0
+#define HINIC_API_CMD_CELL_CTRL_RD_DMA_ATTR_SHIFT               16
+#define HINIC_API_CMD_CELL_CTRL_WR_DMA_ATTR_SHIFT               24
+#define HINIC_API_CMD_CELL_CTRL_XOR_CHKSUM_SHIFT                56
+
+#define HINIC_API_CMD_CELL_CTRL_DATA_SZ_MASK                    0x3F
+#define HINIC_API_CMD_CELL_CTRL_RD_DMA_ATTR_MASK                0x3F
+#define HINIC_API_CMD_CELL_CTRL_WR_DMA_ATTR_MASK                0x3F
+#define HINIC_API_CMD_CELL_CTRL_XOR_CHKSUM_MASK                 0xFF
+
+#define HINIC_API_CMD_CELL_CTRL_SET(val, member)                \
+	((((u64)val) & HINIC_API_CMD_CELL_CTRL_##member##_MASK) << \
+	 HINIC_API_CMD_CELL_CTRL_##member##_SHIFT)
+
+#define HINIC_API_CMD_DESC_API_TYPE_SHIFT                       0
+#define HINIC_API_CMD_DESC_RD_WR_SHIFT                          1
+#define HINIC_API_CMD_DESC_MGMT_BYPASS_SHIFT                    2
+#define HINIC_API_CMD_DESC_DEST_SHIFT                           32
+#define HINIC_API_CMD_DESC_SIZE_SHIFT                           40
+#define HINIC_API_CMD_DESC_XOR_CHKSUM_SHIFT                     56
+
+#define HINIC_API_CMD_DESC_API_TYPE_MASK                        0x1
+#define HINIC_API_CMD_DESC_RD_WR_MASK                           0x1
+#define HINIC_API_CMD_DESC_MGMT_BYPASS_MASK                     0x1
+#define HINIC_API_CMD_DESC_DEST_MASK                            0x1F
+#define HINIC_API_CMD_DESC_SIZE_MASK                            0x7FF
+#define HINIC_API_CMD_DESC_XOR_CHKSUM_MASK                      0xFF
+
+#define HINIC_API_CMD_DESC_SET(val, member)                     \
+	((((u64)val) & HINIC_API_CMD_DESC_##member##_MASK) <<   \
+	 HINIC_API_CMD_DESC_##member##_SHIFT)
+
+#define HINIC_API_CMD_STATUS_HEADER_CHAIN_ID_SHIFT              16
+
+#define HINIC_API_CMD_STATUS_HEADER_CHAIN_ID_MASK               0xFF
+
+#define HINIC_API_CMD_STATUS_HEADER_GET(val, member)            \
+	(((val) >> HINIC_API_CMD_STATUS_HEADER_##member##_SHIFT) & \
+	 HINIC_API_CMD_STATUS_HEADER_##member##_MASK)
+
+#define HINIC_API_CMD_STATUS_CONS_IDX_SHIFT                     0
+#define HINIC_API_CMD_STATUS_CHKSUM_ERR_SHIFT                   28
+
+#define HINIC_API_CMD_STATUS_CONS_IDX_MASK                      0xFFFFFF
+#define HINIC_API_CMD_STATUS_CHKSUM_ERR_MASK                    0x3
+
+#define HINIC_API_CMD_STATUS_GET(val, member)                   \
+	(((val) >> HINIC_API_CMD_STATUS_##member##_SHIFT) &     \
+	 HINIC_API_CMD_STATUS_##member##_MASK)
+
 enum hinic_api_cmd_chain_type {
 	HINIC_API_CMD_WRITE_TO_MGMT_CPU = 2,
 
@@ -122,6 +185,8 @@ struct hinic_api_cmd_chain {
 	u32                             prod_idx;
 	u32                             cons_idx;
 
+	struct semaphore                sem;
+
 	struct hinic_api_cmd_cell_ctxt  *cell_ctxt;
 
 	dma_addr_t                      wb_status_paddr;
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_csr.h b/drivers/net/ethernet/huawei/hinic/hinic_hw_csr.h
index 0c100ee..ebbf054 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_csr.h
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_csr.h
@@ -56,6 +56,13 @@
 #define HINIC_CSR_API_CMD_CHAIN_CTRL_ADDR(idx)          \
 	(HINIC_CSR_API_CMD_BASE + 0x14 + (idx) * HINIC_CSR_API_CMD_STRIDE)
 
+#define HINIC_CSR_API_CMD_CHAIN_PI_ADDR(idx)            \
+	(HINIC_CSR_API_CMD_BASE + 0x1C + (idx) * HINIC_CSR_API_CMD_STRIDE)
+
 #define HINIC_CSR_API_CMD_CHAIN_REQ_ADDR(idx)           \
 	(HINIC_CSR_API_CMD_BASE + 0x20 + (idx) * HINIC_CSR_API_CMD_STRIDE)
+
+#define HINIC_CSR_API_CMD_STATUS_ADDR(idx)              \
+	(HINIC_CSR_API_CMD_BASE + 0x30 + (idx) * HINIC_CSR_API_CMD_STRIDE)
+
 #endif
-- 
1.9.1

[toc] | [prev] | [next] | [standalone]


#1702902 — [PATCH V3 net-next 05/21] net-next/hinic: Add management messages

FromAviad Krawczyk <aviad.krawczyk@huawei.com>
Date2017-08-03 12:20 +0200
Subject[PATCH V3 net-next 05/21] net-next/hinic: Add management messages
Message-ID<ualot-6Yh-9@gated-at.bofh.it>
In reply to#1702870
Add the management messages for sending to api cmd and the
asynchronous event handler for the completion of the messages.

Signed-off-by: Aviad Krawczyk <aviad.krawczyk@huawei.com>
Signed-off-by: Zhao Chen <zhaochen6@huawei.com>
---
 .../net/ethernet/huawei/hinic/hinic_hw_api_cmd.c   |  35 ++
 .../net/ethernet/huawei/hinic/hinic_hw_api_cmd.h   |   3 +
 drivers/net/ethernet/huawei/hinic/hinic_hw_if.h    |   5 +
 drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.c  | 440 ++++++++++++++++++++-
 drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.h  |  59 +++
 5 files changed, 539 insertions(+), 3 deletions(-)

diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.c b/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.c
index 874a262..a99d87c 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.c
@@ -54,6 +54,41 @@ enum api_cmd_xor_chk_level {
 };
 
 /**
+ * api_cmd - API CMD command
+ * @chain: chain for the command
+ * @dest: destination node on the card that will receive the command
+ * @cmd: command data
+ * @size: the command size
+ *
+ * Return 0 - Success, negative - Failure
+ **/
+static int api_cmd(struct hinic_api_cmd_chain *chain,
+		   enum hinic_node_id dest, u8 *cmd, u16 cmd_size)
+{
+	/* should be implemented */
+	return -EINVAL;
+}
+
+/**
+ * hinic_api_cmd_write - Write API CMD command
+ * @chain: chain for write command
+ * @dest: destination node on the card that will receive the command
+ * @cmd: command data
+ * @size: the command size
+ *
+ * Return 0 - Success, negative - Failure
+ **/
+int hinic_api_cmd_write(struct hinic_api_cmd_chain *chain,
+			enum hinic_node_id dest, u8 *cmd, u16 size)
+{
+	/* Verify the chain type */
+	if (chain->chain_type == HINIC_API_CMD_WRITE_TO_MGMT_CPU)
+		return api_cmd(chain, dest, cmd, size);
+
+	return -EINVAL;
+}
+
+/**
  * api_cmd_hw_restart - restart the chain in the HW
  * @chain: the API CMD specific chain to restart
  *
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.h b/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.h
index 0c83b80..e8865d6 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.h
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.h
@@ -132,6 +132,9 @@ struct hinic_api_cmd_chain {
 	struct hinic_api_cmd_cell       *curr_node;
 };
 
+int hinic_api_cmd_write(struct hinic_api_cmd_chain *chain,
+			enum hinic_node_id dest, u8 *cmd, u16 size);
+
 int hinic_api_cmd_init(struct hinic_api_cmd_chain **chain,
 		       struct hinic_hwif *hwif);
 
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_if.h b/drivers/net/ethernet/huawei/hinic/hinic_hw_if.h
index b6d9850..98623d6 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_if.h
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_if.h
@@ -93,6 +93,7 @@
 #define HINIC_HWIF_NUM_IRQS(hwif)       ((hwif)->attr.num_irqs)
 #define HINIC_HWIF_FUNC_IDX(hwif)       ((hwif)->attr.func_idx)
 #define HINIC_HWIF_PCI_INTF(hwif)       ((hwif)->attr.pci_intf_idx)
+#define HINIC_HWIF_PF_IDX(hwif)         ((hwif)->attr.pf_idx)
 
 #define HINIC_FUNC_TYPE(hwif)           ((hwif)->attr.func_type)
 #define HINIC_IS_PF(hwif)               (HINIC_FUNC_TYPE(hwif) == HINIC_PF)
@@ -127,6 +128,10 @@ enum hinic_mod_type {
 	HINIC_MOD_MAX   = 15
 };
 
+enum hinic_node_id {
+	HINIC_NODE_ID_MGMT = 21,
+};
+
 struct hinic_func_attr {
 	u16                     func_idx;
 	u8                      pf_idx;
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.c b/drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.c
index f914bc7..0d079ed 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.c
@@ -16,6 +16,12 @@
 #include <linux/kernel.h>
 #include <linux/types.h>
 #include <linux/errno.h>
+#include <linux/pci.h>
+#include <linux/device.h>
+#include <linux/semaphore.h>
+#include <linux/completion.h>
+#include <linux/slab.h>
+#include <asm/barrier.h>
 
 #include "hinic_hw_if.h"
 #include "hinic_hw_eqs.h"
@@ -23,9 +29,268 @@
 #include "hinic_hw_mgmt.h"
 #include "hinic_hw_dev.h"
 
+#define SYNC_MSG_ID_MASK                0x1FF
+
+#define SYNC_MSG_ID(pf_to_mgmt)         ((pf_to_mgmt)->sync_msg_id)
+
+#define SYNC_MSG_ID_INC(pf_to_mgmt)     (SYNC_MSG_ID(pf_to_mgmt) = \
+					((SYNC_MSG_ID(pf_to_mgmt) + 1) & \
+					 SYNC_MSG_ID_MASK))
+
+#define MSG_SZ_IS_VALID(in_size)        ((in_size) <= MAX_MSG_LEN)
+
+#define MGMT_MSG_LEN_MIN                20
+#define MGMT_MSG_LEN_STEP               16
+#define MGMT_MSG_RSVD_FOR_DEV           8
+
+#define SEGMENT_LEN                     48
+
+#define MAX_PF_MGMT_BUF_SIZE            2048
+
+/* Data should be SEG LEN size aligned */
+#define MAX_MSG_LEN                     2016
+
+#define MSG_NOT_RESP                    0xFFFF
+
+#define MGMT_MSG_TIMEOUT                1000
+
 #define mgmt_to_pfhwdev(pf_mgmt)        \
 		container_of(pf_mgmt, struct hinic_pfhwdev, pf_to_mgmt)
 
+enum msg_segment_type {
+	NOT_LAST_SEGMENT        = 0,
+	LAST_SEGMENT            = 1,
+};
+
+enum mgmt_direction_type {
+	MGMT_DIRECT_SEND        = 0,
+	MGMT_RESP               = 1,
+};
+
+enum msg_ack_type {
+	MSG_ACK         = 0,
+	MSG_NO_ACK      = 1,
+};
+
+/**
+ * prepare_header - prepare the header of the message
+ * @pf_to_mgmt: PF to MGMT channel
+ * @msg_len: the length of the message
+ * @mod: module in the chip that will get the message
+ * @ack_type: ask for response
+ * @direction: the direction of the message
+ * @cmd: command of the message
+ * @msg_id: message id
+ *
+ * Return the prepared header value
+ **/
+static u64 prepare_header(struct hinic_pf_to_mgmt *pf_to_mgmt,
+			  u16 msg_len, enum hinic_mod_type mod,
+			  enum msg_ack_type ack_type,
+			  enum mgmt_direction_type direction,
+			  u16 cmd, u16 msg_id)
+{
+	struct hinic_hwif *hwif = pf_to_mgmt->hwif;
+
+	return HINIC_MSG_HEADER_SET(msg_len, MSG_LEN)           |
+	       HINIC_MSG_HEADER_SET(mod, MODULE)                |
+	       HINIC_MSG_HEADER_SET(SEGMENT_LEN, SEG_LEN)       |
+	       HINIC_MSG_HEADER_SET(ack_type, NO_ACK)           |
+	       HINIC_MSG_HEADER_SET(0, ASYNC_MGMT_TO_PF)        |
+	       HINIC_MSG_HEADER_SET(0, SEQID)                   |
+	       HINIC_MSG_HEADER_SET(LAST_SEGMENT, LAST)         |
+	       HINIC_MSG_HEADER_SET(direction, DIRECTION)       |
+	       HINIC_MSG_HEADER_SET(cmd, CMD)                   |
+	       HINIC_MSG_HEADER_SET(HINIC_HWIF_PCI_INTF(hwif), PCI_INTF) |
+	       HINIC_MSG_HEADER_SET(HINIC_HWIF_PF_IDX(hwif), PF_IDX)     |
+	       HINIC_MSG_HEADER_SET(msg_id, MSG_ID);
+}
+
+/**
+ * prepare_mgmt_cmd - prepare the mgmt command
+ * @mgmt_cmd: pointer to the command to prepare
+ * @header: pointer of the header for the message
+ * @msg: the data of the message
+ * @msg_len: the length of the message
+ **/
+static void prepare_mgmt_cmd(u8 *mgmt_cmd, u64 *header, u8 *msg, u16 msg_len)
+{
+	memset(mgmt_cmd, 0, MGMT_MSG_RSVD_FOR_DEV);
+
+	mgmt_cmd += MGMT_MSG_RSVD_FOR_DEV;
+	memcpy(mgmt_cmd, header, sizeof(*header));
+
+	mgmt_cmd += sizeof(*header);
+	memcpy(mgmt_cmd, msg, msg_len);
+}
+
+/**
+ * mgmt_msg_len - calculate the total message length
+ * @msg_data_len: the length of the message data
+ *
+ * Return the total message length
+ **/
+static u16 mgmt_msg_len(u16 msg_data_len)
+{
+	/* RSVD + HEADER_SIZE + DATA_LEN */
+	u16 msg_len = MGMT_MSG_RSVD_FOR_DEV + sizeof(u64) + msg_data_len;
+
+	if (msg_len > MGMT_MSG_LEN_MIN)
+		msg_len = MGMT_MSG_LEN_MIN +
+			   ALIGN((msg_len - MGMT_MSG_LEN_MIN),
+				 MGMT_MSG_LEN_STEP);
+	else
+		msg_len = MGMT_MSG_LEN_MIN;
+
+	return msg_len;
+}
+
+/**
+ * send_msg_to_mgmt - send message to mgmt by API CMD
+ * @pf_to_mgmt: PF to MGMT channel
+ * @mod: module in the chip that will get the message
+ * @cmd: command of the message
+ * @data: the msg data
+ * @data_len: the msg data length
+ * @ack_type: ask for response
+ * @direction: the direction of the original message
+ * @resp_msg_id: msg id to response for
+ *
+ * Return 0 - Success, negative - Failure
+ **/
+static int send_msg_to_mgmt(struct hinic_pf_to_mgmt *pf_to_mgmt,
+			    enum hinic_mod_type mod, u8 cmd,
+			    u8 *data, u16 data_len,
+			    enum msg_ack_type ack_type,
+			    enum mgmt_direction_type direction,
+			    u16 resp_msg_id)
+{
+	struct hinic_api_cmd_chain *chain;
+	u8 *mgmt_cmd = pf_to_mgmt->sync_msg_buf;
+	u64 header;
+	u16 msg_id, cmd_size = mgmt_msg_len(data_len);
+
+	msg_id = SYNC_MSG_ID(pf_to_mgmt);
+
+	if (direction == MGMT_RESP) {
+		header = prepare_header(pf_to_mgmt, data_len, mod, ack_type,
+					direction, cmd, resp_msg_id);
+	} else {
+		SYNC_MSG_ID_INC(pf_to_mgmt);
+		header = prepare_header(pf_to_mgmt, data_len, mod, ack_type,
+					direction, cmd, msg_id);
+	}
+
+	prepare_mgmt_cmd(mgmt_cmd, &header, data, data_len);
+
+	chain = pf_to_mgmt->cmd_chain[HINIC_API_CMD_WRITE_TO_MGMT_CPU];
+
+	return hinic_api_cmd_write(chain, HINIC_NODE_ID_MGMT, mgmt_cmd,
+				   cmd_size);
+}
+
+/**
+ * msg_to_mgmt_sync - send sync message to mgmt
+ * @pf_to_mgmt: PF to MGMT channel
+ * @mod: module in the chip that will get the message
+ * @cmd: command of the message
+ * @buf_in: the msg data
+ * @in_size: the msg data length
+ * @buf_out: response
+ * @out_size: response length
+ * @direction: the direction of the original message
+ * @resp_msg_id: msg id to response for
+ *
+ * Return 0 - Success, negative - Failure
+ **/
+static int msg_to_mgmt_sync(struct hinic_pf_to_mgmt *pf_to_mgmt,
+			    enum hinic_mod_type mod, u8 cmd,
+			    u8 *buf_in, u16 in_size,
+			    u8 *buf_out, u16 *out_size,
+			    enum mgmt_direction_type direction,
+			    u16 resp_msg_id)
+{
+	struct hinic_hwif *hwif = pf_to_mgmt->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+	struct hinic_recv_msg *recv_msg;
+	struct completion *recv_done;
+	u16 msg_id;
+	int err;
+
+	/* Lock the sync_msg_buf */
+	down(&pf_to_mgmt->sync_msg_lock);
+
+	recv_msg = &pf_to_mgmt->recv_resp_msg_from_mgmt;
+	recv_done = &recv_msg->recv_done;
+
+	if (resp_msg_id == MSG_NOT_RESP)
+		msg_id = SYNC_MSG_ID(pf_to_mgmt);
+	else
+		msg_id = resp_msg_id;
+
+	init_completion(recv_done);
+
+	err = send_msg_to_mgmt(pf_to_mgmt, mod, cmd, buf_in, in_size,
+			       MSG_ACK, direction, resp_msg_id);
+	if (err) {
+		dev_err(&pdev->dev, "Failed to send sync msg to mgmt\n");
+		goto unlock_sync_msg;
+	}
+
+	if (!wait_for_completion_timeout(recv_done, MGMT_MSG_TIMEOUT)) {
+		dev_err(&pdev->dev, "MGMT timeout, MSG id = %d\n", msg_id);
+		err = -ETIMEDOUT;
+		goto unlock_sync_msg;
+	}
+
+	smp_rmb();      /* verify reading after completion */
+
+	if (recv_msg->msg_id != msg_id) {
+		dev_err(&pdev->dev, "incorrect MSG for id = %d\n", msg_id);
+		err = -EFAULT;
+		goto unlock_sync_msg;
+	}
+
+	if ((buf_out) && (recv_msg->msg_len <= MAX_PF_MGMT_BUF_SIZE)) {
+		memcpy(buf_out, recv_msg->msg, recv_msg->msg_len);
+		*out_size = recv_msg->msg_len;
+	}
+
+unlock_sync_msg:
+	up(&pf_to_mgmt->sync_msg_lock);
+	return err;
+}
+
+/**
+ * msg_to_mgmt_async - send message to mgmt without response
+ * @pf_to_mgmt: PF to MGMT channel
+ * @mod: module in the chip that will get the message
+ * @cmd: command of the message
+ * @buf_in: the msg data
+ * @in_size: the msg data length
+ * @direction: the direction of the original message
+ * @resp_msg_id: msg id to response for
+ *
+ * Return 0 - Success, negative - Failure
+ **/
+static int msg_to_mgmt_async(struct hinic_pf_to_mgmt *pf_to_mgmt,
+			     enum hinic_mod_type mod, u8 cmd,
+			     u8 *buf_in, u16 in_size,
+			     enum mgmt_direction_type direction,
+			     u16 resp_msg_id)
+{
+	int err;
+
+	/* Lock the sync_msg_buf */
+	down(&pf_to_mgmt->sync_msg_lock);
+
+	err = send_msg_to_mgmt(pf_to_mgmt, mod, cmd, buf_in, in_size,
+			       MSG_NO_ACK, direction, resp_msg_id);
+
+	up(&pf_to_mgmt->sync_msg_lock);
+	return err;
+}
+
 /**
  * hinic_msg_to_mgmt - send message to mgmt
  * @pf_to_mgmt: PF to MGMT channel
@@ -44,8 +309,98 @@ int hinic_msg_to_mgmt(struct hinic_pf_to_mgmt *pf_to_mgmt,
 		      void *buf_in, u16 in_size, void *buf_out, u16 *out_size,
 		      enum hinic_mgmt_msg_type sync)
 {
-	/* should be implemented */
-	return -EINVAL;
+	struct hinic_hwif *hwif = pf_to_mgmt->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+
+	if (sync != HINIC_MGMT_MSG_SYNC) {
+		dev_err(&pdev->dev, "Invalid MGMT msg type\n");
+		return -EINVAL;
+	}
+
+	if (!MSG_SZ_IS_VALID(in_size)) {
+		dev_err(&pdev->dev, "Invalid MGMT msg buffer size\n");
+		return -EINVAL;
+	}
+
+	return msg_to_mgmt_sync(pf_to_mgmt, mod, cmd, buf_in, in_size,
+				buf_out, out_size, MGMT_DIRECT_SEND,
+				MSG_NOT_RESP);
+}
+
+/**
+ * mgmt_recv_msg_handler - handler for message from mgmt cpu
+ * @pf_to_mgmt: PF to MGMT channel
+ * @recv_msg: received message details
+ **/
+static void mgmt_recv_msg_handler(struct hinic_pf_to_mgmt *pf_to_mgmt,
+				  struct hinic_recv_msg *recv_msg)
+{
+	struct hinic_hwif *hwif = pf_to_mgmt->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+	u8 *buf_out = recv_msg->buf_out;
+	enum hinic_mod_type mod = recv_msg->mod;
+	u16 out_size = 0;
+
+	dev_err(&pdev->dev, "No MGMT msg handler, mod = %d\n", mod);
+
+	if (!recv_msg->async_mgmt_to_pf)
+		/* MGMT sent sync msg, send the response */
+		msg_to_mgmt_async(pf_to_mgmt, mod, recv_msg->cmd,
+				  buf_out, out_size, MGMT_RESP,
+				  recv_msg->msg_id);
+}
+
+/**
+ * mgmt_resp_msg_handler - handler for a response message from mgmt cpu
+ * @pf_to_mgmt: PF to MGMT channel
+ * @recv_msg: received message details
+ **/
+static void mgmt_resp_msg_handler(struct hinic_pf_to_mgmt *pf_to_mgmt,
+				  struct hinic_recv_msg *recv_msg)
+{
+	wmb();  /* verify writing all, before reading */
+
+	complete(&recv_msg->recv_done);
+}
+
+/**
+ * recv_mgmt_msg_handler - handler for a message from mgmt cpu
+ * @pf_to_mgmt: PF to MGMT channel
+ * @header: the header of the message
+ * @recv_msg: received message details
+ **/
+static void recv_mgmt_msg_handler(struct hinic_pf_to_mgmt *pf_to_mgmt,
+				  u64 *header, struct hinic_recv_msg *recv_msg)
+{
+	struct hinic_hwif *hwif = pf_to_mgmt->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+	u8 *msg_body = (u8 *)header + sizeof(*header);
+	int seq_id, seg_len;
+
+	seq_id = HINIC_MSG_HEADER_GET(*header, SEQID);
+	seg_len = HINIC_MSG_HEADER_GET(*header, SEG_LEN);
+
+	if (seq_id >= (MAX_MSG_LEN / SEGMENT_LEN)) {
+		dev_err(&pdev->dev, "recv big mgmt msg\n");
+		return;
+	}
+
+	memcpy(recv_msg->msg + seq_id * SEGMENT_LEN, msg_body, seg_len);
+
+	if (!HINIC_MSG_HEADER_GET(*header, LAST))
+		return;
+
+	recv_msg->cmd = HINIC_MSG_HEADER_GET(*header, CMD);
+	recv_msg->mod = HINIC_MSG_HEADER_GET(*header, MODULE);
+	recv_msg->async_mgmt_to_pf = HINIC_MSG_HEADER_GET(*header,
+							  ASYNC_MGMT_TO_PF);
+	recv_msg->msg_len = HINIC_MSG_HEADER_GET(*header, MSG_LEN);
+	recv_msg->msg_id = HINIC_MSG_HEADER_GET(*header, MSG_ID);
+
+	if (HINIC_MSG_HEADER_GET(*header, DIRECTION) == MGMT_RESP)
+		mgmt_resp_msg_handler(pf_to_mgmt, recv_msg);
+	else
+		mgmt_recv_msg_handler(pf_to_mgmt, recv_msg);
 }
 
 /**
@@ -56,7 +411,77 @@ int hinic_msg_to_mgmt(struct hinic_pf_to_mgmt *pf_to_mgmt,
  **/
 static void mgmt_msg_aeqe_handler(void *handle, void *data, u8 size)
 {
-	/* should be implemented */
+	struct hinic_pf_to_mgmt *pf_to_mgmt = handle;
+	struct hinic_recv_msg *recv_msg;
+	u64 *header = (u64 *)data;
+
+	recv_msg = HINIC_MSG_HEADER_GET(*header, DIRECTION) ==
+		   MGMT_DIRECT_SEND ?
+		   &pf_to_mgmt->recv_msg_from_mgmt :
+		   &pf_to_mgmt->recv_resp_msg_from_mgmt;
+
+	recv_mgmt_msg_handler(pf_to_mgmt, header, recv_msg);
+}
+
+/**
+ * alloc_recv_msg - allocate receive message memory
+ * @pf_to_mgmt: PF to MGMT channel
+ * @recv_msg: pointer that will hold the allocated data
+ *
+ * Return 0 - Success, negative - Failure
+ **/
+static int alloc_recv_msg(struct hinic_pf_to_mgmt *pf_to_mgmt,
+			  struct hinic_recv_msg *recv_msg)
+{
+	struct hinic_hwif *hwif = pf_to_mgmt->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+
+	recv_msg->msg = devm_kzalloc(&pdev->dev, MAX_PF_MGMT_BUF_SIZE,
+				     GFP_KERNEL);
+	if (!recv_msg->msg)
+		return -ENOMEM;
+
+	recv_msg->buf_out = devm_kzalloc(&pdev->dev, MAX_PF_MGMT_BUF_SIZE,
+					 GFP_KERNEL);
+	if (!recv_msg->buf_out)
+		return -ENOMEM;
+
+	return 0;
+}
+
+/**
+ * alloc_msg_buf - allocate all the message buffers of PF to MGMT channel
+ * @pf_to_mgmt: PF to MGMT channel
+ *
+ * Return 0 - Success, negative - Failure
+ **/
+static int alloc_msg_buf(struct hinic_pf_to_mgmt *pf_to_mgmt)
+{
+	struct hinic_hwif *hwif = pf_to_mgmt->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+	int err;
+
+	err = alloc_recv_msg(pf_to_mgmt,
+			     &pf_to_mgmt->recv_msg_from_mgmt);
+	if (err) {
+		dev_err(&pdev->dev, "Failed to allocate recv msg\n");
+		return err;
+	}
+
+	err = alloc_recv_msg(pf_to_mgmt,
+			     &pf_to_mgmt->recv_resp_msg_from_mgmt);
+	if (err) {
+		dev_err(&pdev->dev, "Failed to allocate resp recv msg\n");
+		return err;
+	}
+
+	pf_to_mgmt->sync_msg_buf = devm_kzalloc(&pdev->dev,
+						MAX_PF_MGMT_BUF_SIZE,
+						GFP_KERNEL);
+	if (!pf_to_mgmt->sync_msg_buf)
+		return -ENOMEM;
+
+	return 0;
 }
 
 /**
@@ -76,6 +501,15 @@ int hinic_pf_to_mgmt_init(struct hinic_pf_to_mgmt *pf_to_mgmt,
 
 	pf_to_mgmt->hwif = hwif;
 
+	sema_init(&pf_to_mgmt->sync_msg_lock, 1);
+	pf_to_mgmt->sync_msg_id = 0;
+
+	err = alloc_msg_buf(pf_to_mgmt);
+	if (err) {
+		dev_err(&pdev->dev, "Failed to allocate msg buffers\n");
+		return err;
+	}
+
 	err = hinic_api_cmd_init(pf_to_mgmt->cmd_chain, hwif);
 	if (err) {
 		dev_err(&pdev->dev, "Failed to initialize cmd chains\n");
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.h b/drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.h
index dff321c..eca7ad8 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.h
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.h
@@ -17,10 +17,48 @@
 #define HINIC_HW_MGMT_H
 
 #include <linux/types.h>
+#include <linux/semaphore.h>
+#include <linux/completion.h>
 
 #include "hinic_hw_if.h"
 #include "hinic_hw_api_cmd.h"
 
+#define HINIC_MSG_HEADER_MSG_LEN_SHIFT                          0
+#define HINIC_MSG_HEADER_MODULE_SHIFT                           11
+#define HINIC_MSG_HEADER_SEG_LEN_SHIFT                          16
+#define HINIC_MSG_HEADER_NO_ACK_SHIFT                           22
+#define HINIC_MSG_HEADER_ASYNC_MGMT_TO_PF_SHIFT                 23
+#define HINIC_MSG_HEADER_SEQID_SHIFT                            24
+#define HINIC_MSG_HEADER_LAST_SHIFT                             30
+#define HINIC_MSG_HEADER_DIRECTION_SHIFT                        31
+#define HINIC_MSG_HEADER_CMD_SHIFT                              32
+#define HINIC_MSG_HEADER_ZEROS_SHIFT                            40
+#define HINIC_MSG_HEADER_PCI_INTF_SHIFT                         48
+#define HINIC_MSG_HEADER_PF_IDX_SHIFT                           50
+#define HINIC_MSG_HEADER_MSG_ID_SHIFT                           54
+
+#define HINIC_MSG_HEADER_MSG_LEN_MASK                           0x7FF
+#define HINIC_MSG_HEADER_MODULE_MASK                            0x1F
+#define HINIC_MSG_HEADER_SEG_LEN_MASK                           0x3F
+#define HINIC_MSG_HEADER_NO_ACK_MASK                            0x1
+#define HINIC_MSG_HEADER_ASYNC_MGMT_TO_PF_MASK                  0x1
+#define HINIC_MSG_HEADER_SEQID_MASK                             0x3F
+#define HINIC_MSG_HEADER_LAST_MASK                              0x1
+#define HINIC_MSG_HEADER_DIRECTION_MASK                         0x1
+#define HINIC_MSG_HEADER_CMD_MASK                               0xFF
+#define HINIC_MSG_HEADER_ZEROS_MASK                             0xFF
+#define HINIC_MSG_HEADER_PCI_INTF_MASK                          0x3
+#define HINIC_MSG_HEADER_PF_IDX_MASK                            0xF
+#define HINIC_MSG_HEADER_MSG_ID_MASK                            0x3FF
+
+#define HINIC_MSG_HEADER_SET(val, member)                       \
+		((u64)((val) & HINIC_MSG_HEADER_##member##_MASK) << \
+		 HINIC_MSG_HEADER_##member##_SHIFT)
+
+#define HINIC_MSG_HEADER_GET(val, member)                       \
+		(((val) >> HINIC_MSG_HEADER_##member##_SHIFT) & \
+		 HINIC_MSG_HEADER_##member##_MASK)
+
 enum hinic_mgmt_msg_type {
 	HINIC_MGMT_MSG_SYNC = 1,
 };
@@ -29,9 +67,30 @@ enum hinic_cfg_cmd {
 	HINIC_CFG_NIC_CAP = 0,
 };
 
+struct hinic_recv_msg {
+	u8                      *msg;
+	u8                      *buf_out;
+
+	struct completion       recv_done;
+
+	u16                     cmd;
+	enum hinic_mod_type     mod;
+	int                     async_mgmt_to_pf;
+
+	u16                     msg_len;
+	u16                     msg_id;
+};
+
 struct hinic_pf_to_mgmt {
 	struct hinic_hwif               *hwif;
 
+	struct semaphore                sync_msg_lock;
+	u16                             sync_msg_id;
+	u8                              *sync_msg_buf;
+
+	struct hinic_recv_msg           recv_resp_msg_from_mgmt;
+	struct hinic_recv_msg           recv_msg_from_mgmt;
+
 	struct hinic_api_cmd_chain      *cmd_chain[HINIC_API_CMD_MAX];
 };
 
-- 
1.9.1

[toc] | [prev] | [next] | [standalone]


#1702908 — [PATCH V3 net-next 03/21] net-next/hinic: Initialize api cmd resources

FromAviad Krawczyk <aviad.krawczyk@huawei.com>
Date2017-08-03 12:20 +0200
Subject[PATCH V3 net-next 03/21] net-next/hinic: Initialize api cmd resources
Message-ID<ualou-6Yh-25@gated-at.bofh.it>
In reply to#1702870
Initialize api cmd resources as part of management initialization.

Signed-off-by: Aviad Krawczyk <aviad.krawczyk@huawei.com>
Signed-off-by: Zhao Chen <zhaochen6@huawei.com>
---
 drivers/net/ethernet/huawei/hinic/Makefile         |   4 +-
 .../net/ethernet/huawei/hinic/hinic_hw_api_cmd.c   | 444 +++++++++++++++++++++
 .../net/ethernet/huawei/hinic/hinic_hw_api_cmd.h   | 102 +++++
 drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.c  |  10 +
 drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.h  |   3 +
 5 files changed, 561 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.c
 create mode 100644 drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.h

diff --git a/drivers/net/ethernet/huawei/hinic/Makefile b/drivers/net/ethernet/huawei/hinic/Makefile
index 717ad71..beba90a 100644
--- a/drivers/net/ethernet/huawei/hinic/Makefile
+++ b/drivers/net/ethernet/huawei/hinic/Makefile
@@ -1,4 +1,4 @@
 obj-$(CONFIG_HINIC) += hinic.o
 
-hinic-y := hinic_main.o hinic_hw_dev.o hinic_hw_mgmt.o hinic_hw_eqs.o \
-	   hinic_hw_if.o
+hinic-y := hinic_main.o hinic_hw_dev.o hinic_hw_mgmt.o hinic_hw_api_cmd.o \
+	   hinic_hw_eqs.o hinic_hw_if.o
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.c b/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.c
new file mode 100644
index 0000000..b2f6bd8
--- /dev/null
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.c
@@ -0,0 +1,444 @@
+/*
+ * Huawei HiNIC PCI Express Linux driver
+ * Copyright(c) 2017 Huawei Technologies Co., Ltd
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+ * for more details.
+ *
+ */
+
+#include <linux/types.h>
+#include <linux/errno.h>
+#include <linux/pci.h>
+#include <linux/device.h>
+#include <linux/slab.h>
+#include <linux/dma-mapping.h>
+#include <linux/bitops.h>
+#include <linux/err.h>
+#include <asm/byteorder.h>
+
+#include "hinic_hw_if.h"
+#include "hinic_hw_api_cmd.h"
+
+#define API_CHAIN_NUM_CELLS                     32
+
+#define API_CMD_CELL_SIZE_SHIFT                 6
+#define API_CMD_CELL_SIZE_MIN                   (BIT(API_CMD_CELL_SIZE_SHIFT))
+
+#define API_CMD_CELL_SIZE(cell_size)            \
+		(((cell_size) >= API_CMD_CELL_SIZE_MIN) ? \
+		 (1 << (fls(cell_size - 1))) : API_CMD_CELL_SIZE_MIN)
+
+#define API_CMD_BUF_SIZE                        2048
+
+/**
+ * api_cmd_chain_hw_init - initialize the chain in the HW
+ * @chain: the API CMD specific chain to initialize in HW
+ *
+ * Return 0 - Success, negative - Failure
+ **/
+static int api_cmd_chain_hw_init(struct hinic_api_cmd_chain *chain)
+{
+	/* should be implemented */
+	return 0;
+}
+
+/**
+ * free_cmd_buf - free the dma buffer of API CMD command
+ * @chain: the API CMD specific chain of the cmd
+ * @cell_idx: the cell index of the cmd
+ **/
+static void free_cmd_buf(struct hinic_api_cmd_chain *chain, int cell_idx)
+{
+	struct hinic_api_cmd_cell_ctxt *cell_ctxt;
+	struct hinic_hwif *hwif = chain->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+
+	cell_ctxt = &chain->cell_ctxt[cell_idx];
+
+	dma_free_coherent(&pdev->dev, API_CMD_BUF_SIZE,
+			  cell_ctxt->api_cmd_vaddr,
+			  cell_ctxt->api_cmd_paddr);
+}
+
+/**
+ * alloc_cmd_buf - allocate a dma buffer for API CMD command
+ * @chain: the API CMD specific chain for the cmd
+ * @cell: the cell in the HW for the cmd
+ * @cell_idx: the index of the cell
+ *
+ * Return 0 - Success, negative - Failure
+ **/
+static int alloc_cmd_buf(struct hinic_api_cmd_chain *chain,
+			 struct hinic_api_cmd_cell *cell, int cell_idx)
+{
+	struct hinic_hwif *hwif = chain->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+	struct hinic_api_cmd_cell_ctxt *cell_ctxt;
+	dma_addr_t cmd_paddr;
+	u8 *cmd_vaddr;
+	int err = 0;
+
+	cmd_vaddr = dma_zalloc_coherent(&pdev->dev, API_CMD_BUF_SIZE,
+					&cmd_paddr, GFP_KERNEL);
+	if (!cmd_vaddr) {
+		dev_err(&pdev->dev, "Failed to allocate API CMD DMA memory\n");
+		return -ENOMEM;
+	}
+
+	cell_ctxt = &chain->cell_ctxt[cell_idx];
+
+	cell_ctxt->api_cmd_vaddr = cmd_vaddr;
+	cell_ctxt->api_cmd_paddr = cmd_paddr;
+
+	/* set the cmd DMA address in the cell */
+	switch (chain->chain_type) {
+	case HINIC_API_CMD_WRITE_TO_MGMT_CPU:
+		/* The data in the HW should be in Big Endian Format */
+		cell->write.hw_cmd_paddr = cpu_to_be64(cmd_paddr);
+		break;
+
+	default:
+		dev_err(&pdev->dev, "Unsupported API CMD chain type\n");
+		free_cmd_buf(chain, cell_idx);
+		err = -EINVAL;
+		break;
+	}
+
+	return err;
+}
+
+/**
+ * api_cmd_create_cell - create API CMD cell for specific chain
+ * @chain: the API CMD specific chain to create its cell
+ * @cell_idx: the index of the cell to create
+ * @pre_node: previous cell
+ * @node_vaddr: the returned virt addr of the cell
+ *
+ * Return 0 - Success, negative - Failure
+ **/
+static int api_cmd_create_cell(struct hinic_api_cmd_chain *chain,
+			       int cell_idx,
+			       struct hinic_api_cmd_cell *pre_node,
+			       struct hinic_api_cmd_cell **node_vaddr)
+{
+	struct hinic_hwif *hwif = chain->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+	struct hinic_api_cmd_cell_ctxt *cell_ctxt;
+	struct hinic_api_cmd_cell *node;
+	dma_addr_t node_paddr;
+	int err;
+
+	node = dma_zalloc_coherent(&pdev->dev, chain->cell_size,
+				   &node_paddr, GFP_KERNEL);
+	if (!node) {
+		dev_err(&pdev->dev, "Failed to allocate dma API CMD cell\n");
+		return -ENOMEM;
+	}
+
+	node->read.hw_wb_resp_paddr = 0;
+
+	cell_ctxt = &chain->cell_ctxt[cell_idx];
+	cell_ctxt->cell_vaddr = node;
+	cell_ctxt->cell_paddr = node_paddr;
+
+	if (!pre_node) {
+		chain->head_cell_paddr = node_paddr;
+		chain->head_node = node;
+	} else {
+		/* The data in the HW should be in Big Endian Format */
+		pre_node->next_cell_paddr = cpu_to_be64(node_paddr);
+	}
+
+	switch (chain->chain_type) {
+	case HINIC_API_CMD_WRITE_TO_MGMT_CPU:
+		err = alloc_cmd_buf(chain, node, cell_idx);
+		if (err) {
+			dev_err(&pdev->dev, "Failed to allocate cmd buffer\n");
+			goto err_alloc_cmd_buf;
+		}
+		break;
+
+	default:
+		dev_err(&pdev->dev, "Unsupported API CMD chain type\n");
+		err = -EINVAL;
+		goto err_alloc_cmd_buf;
+	}
+
+	*node_vaddr = node;
+	return 0;
+
+err_alloc_cmd_buf:
+	dma_free_coherent(&pdev->dev, chain->cell_size, node, node_paddr);
+	return err;
+}
+
+/**
+ * api_cmd_destroy_cell - destroy API CMD cell of specific chain
+ * @chain: the API CMD specific chain to destroy its cell
+ * @cell_idx: the cell to destroy
+ **/
+static void api_cmd_destroy_cell(struct hinic_api_cmd_chain *chain,
+				 int cell_idx)
+{
+	struct hinic_hwif *hwif = chain->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+	struct hinic_api_cmd_cell_ctxt *cell_ctxt;
+	struct hinic_api_cmd_cell *node;
+	dma_addr_t node_paddr;
+	size_t node_size;
+
+	cell_ctxt = &chain->cell_ctxt[cell_idx];
+
+	node = cell_ctxt->cell_vaddr;
+	node_paddr = cell_ctxt->cell_paddr;
+	node_size = chain->cell_size;
+
+	if (cell_ctxt->api_cmd_vaddr) {
+		switch (chain->chain_type) {
+		case HINIC_API_CMD_WRITE_TO_MGMT_CPU:
+			free_cmd_buf(chain, cell_idx);
+			break;
+		default:
+			dev_err(&pdev->dev, "Unsupported API CMD chain type\n");
+			break;
+		}
+
+		dma_free_coherent(&pdev->dev, node_size, node,
+				  node_paddr);
+	}
+}
+
+/**
+ * api_cmd_destroy_cells - destroy API CMD cells of specific chain
+ * @chain: the API CMD specific chain to destroy its cells
+ * @num_cells: number of cells to destroy
+ **/
+static void api_cmd_destroy_cells(struct hinic_api_cmd_chain *chain,
+				  int num_cells)
+{
+	int cell_idx;
+
+	for (cell_idx = 0; cell_idx < num_cells; cell_idx++)
+		api_cmd_destroy_cell(chain, cell_idx);
+}
+
+/**
+ * api_cmd_create_cells - create API CMD cells for specific chain
+ * @chain: the API CMD specific chain
+ *
+ * Return 0 - Success, negative - Failure
+ **/
+static int api_cmd_create_cells(struct hinic_api_cmd_chain *chain)
+{
+	struct hinic_api_cmd_cell *node = NULL, *pre_node = NULL;
+	struct hinic_hwif *hwif = chain->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+	int err, cell_idx;
+
+	for (cell_idx = 0; cell_idx < chain->num_cells; cell_idx++) {
+		err = api_cmd_create_cell(chain, cell_idx, pre_node, &node);
+		if (err) {
+			dev_err(&pdev->dev, "Failed to create API CMD cell\n");
+			goto err_create_cell;
+		}
+
+		pre_node = node;
+	}
+
+	/* set the Final node to point on the start */
+	node->next_cell_paddr = cpu_to_be64(chain->head_cell_paddr);
+
+	/* set the current node to be the head */
+	chain->curr_node = chain->head_node;
+	return 0;
+
+err_create_cell:
+	api_cmd_destroy_cells(chain, cell_idx);
+	return err;
+}
+
+/**
+ * api_chain_init - initialize API CMD specific chain
+ * @chain: the API CMD specific chain to initialize
+ * @attr: attributes to set in the chain
+ *
+ * Return 0 - Success, negative - Failure
+ **/
+static int api_chain_init(struct hinic_api_cmd_chain *chain,
+			  struct hinic_api_cmd_chain_attr *attr)
+{
+	struct hinic_hwif *hwif = attr->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+	size_t cell_ctxt_size;
+
+	chain->hwif = hwif;
+	chain->chain_type  = attr->chain_type;
+	chain->num_cells = attr->num_cells;
+	chain->cell_size = attr->cell_size;
+
+	chain->prod_idx  = 0;
+	chain->cons_idx  = 0;
+
+	cell_ctxt_size = chain->num_cells * sizeof(*chain->cell_ctxt);
+	chain->cell_ctxt = devm_kzalloc(&pdev->dev, cell_ctxt_size, GFP_KERNEL);
+	if (!chain->cell_ctxt)
+		return -ENOMEM;
+
+	chain->wb_status = dma_zalloc_coherent(&pdev->dev,
+					       sizeof(*chain->wb_status),
+					       &chain->wb_status_paddr,
+					       GFP_KERNEL);
+	if (!chain->wb_status) {
+		dev_err(&pdev->dev, "Failed to allocate DMA wb status\n");
+		return -ENOMEM;
+	}
+
+	return 0;
+}
+
+/**
+ * api_chain_free - free API CMD specific chain
+ * @chain: the API CMD specific chain to free
+ **/
+static void api_chain_free(struct hinic_api_cmd_chain *chain)
+{
+	struct hinic_hwif *hwif = chain->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+
+	dma_free_coherent(&pdev->dev, sizeof(*chain->wb_status),
+			  chain->wb_status, chain->wb_status_paddr);
+}
+
+/**
+ * api_cmd_create_chain - create API CMD specific chain
+ * @attr: attributes to set the chain
+ *
+ * Return the created chain
+ **/
+static struct hinic_api_cmd_chain *
+	api_cmd_create_chain(struct hinic_api_cmd_chain_attr *attr)
+{
+	struct hinic_hwif *hwif = attr->hwif;
+	struct pci_dev *pdev = hwif->pdev;
+	struct hinic_api_cmd_chain *chain;
+	int err;
+
+	if (attr->num_cells & (attr->num_cells - 1)) {
+		dev_err(&pdev->dev, "Invalid number of cells, must be power of 2\n");
+		return ERR_PTR(-EINVAL);
+	}
+
+	chain = devm_kzalloc(&pdev->dev, sizeof(*chain), GFP_KERNEL);
+	if (!chain)
+		return ERR_PTR(-ENOMEM);
+
+	err = api_chain_init(chain, attr);
+	if (err) {
+		dev_err(&pdev->dev, "Failed to initialize chain\n");
+		return ERR_PTR(err);
+	}
+
+	err = api_cmd_create_cells(chain);
+	if (err) {
+		dev_err(&pdev->dev, "Failed to create cells for API CMD chain\n");
+		goto err_create_cells;
+	}
+
+	err = api_cmd_chain_hw_init(chain);
+	if (err) {
+		dev_err(&pdev->dev, "Failed to initialize chain HW\n");
+		goto err_chain_hw_init;
+	}
+
+	return chain;
+
+err_chain_hw_init:
+	api_cmd_destroy_cells(chain, chain->num_cells);
+
+err_create_cells:
+	api_chain_free(chain);
+	return ERR_PTR(err);
+}
+
+/**
+ * api_cmd_destroy_chain - destroy API CMD specific chain
+ * @chain: the API CMD specific chain to destroy
+ **/
+static void api_cmd_destroy_chain(struct hinic_api_cmd_chain *chain)
+{
+	api_cmd_destroy_cells(chain, chain->num_cells);
+	api_chain_free(chain);
+}
+
+/**
+ * hinic_api_cmd_init - Initialize all the API CMD chains
+ * @chain: the API CMD chains that are initialized
+ * @hwif: the hardware interface of a pci function device
+ *
+ * Return 0 - Success, negative - Failure
+ **/
+int hinic_api_cmd_init(struct hinic_api_cmd_chain **chain,
+		       struct hinic_hwif *hwif)
+{
+	struct pci_dev *pdev = hwif->pdev;
+	struct hinic_api_cmd_chain_attr attr;
+	enum hinic_api_cmd_chain_type type, chain_type;
+	size_t hw_cell_sz = sizeof(struct hinic_api_cmd_cell);
+	int err;
+
+	attr.hwif = hwif;
+	attr.num_cells  = API_CHAIN_NUM_CELLS;
+	attr.cell_size  = API_CMD_CELL_SIZE(hw_cell_sz);
+
+	chain_type = HINIC_API_CMD_WRITE_TO_MGMT_CPU;
+	for ( ; chain_type < HINIC_API_CMD_MAX; chain_type++) {
+		attr.chain_type = chain_type;
+
+		if (chain_type != HINIC_API_CMD_WRITE_TO_MGMT_CPU)
+			continue;
+
+		chain[chain_type] = api_cmd_create_chain(&attr);
+		if (IS_ERR(chain[chain_type])) {
+			dev_err(&pdev->dev, "Failed to create chain %d\n",
+				chain_type);
+			goto err_create_chain;
+		}
+	}
+
+	return 0;
+
+err_create_chain:
+	type = HINIC_API_CMD_WRITE_TO_MGMT_CPU;
+	for ( ; type < chain_type; type++) {
+		if (type != HINIC_API_CMD_WRITE_TO_MGMT_CPU)
+			continue;
+
+		api_cmd_destroy_chain(chain[type]);
+	}
+
+	return err;
+}
+
+/**
+ * hinic_api_cmd_free - free the API CMD chains
+ * @chain: the API CMD chains that are freed
+ **/
+void hinic_api_cmd_free(struct hinic_api_cmd_chain **chain)
+{
+	enum hinic_api_cmd_chain_type chain_type;
+
+	chain_type = HINIC_API_CMD_WRITE_TO_MGMT_CPU;
+	for ( ; chain_type < HINIC_API_CMD_MAX; chain_type++) {
+		if (chain_type != HINIC_API_CMD_WRITE_TO_MGMT_CPU)
+			continue;
+
+		api_cmd_destroy_chain(chain[chain_type]);
+	}
+}
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.h b/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.h
new file mode 100644
index 0000000..9c7d1e5
--- /dev/null
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.h
@@ -0,0 +1,102 @@
+/*
+ * Huawei HiNIC PCI Express Linux driver
+ * Copyright(c) 2017 Huawei Technologies Co., Ltd
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+ * for more details.
+ *
+ */
+
+#ifndef HINIC_HW_API_CMD_H
+#define HINIC_HW_API_CMD_H
+
+#include <linux/types.h>
+
+#include "hinic_hw_if.h"
+
+enum hinic_api_cmd_chain_type {
+	HINIC_API_CMD_WRITE_TO_MGMT_CPU = 2,
+
+	HINIC_API_CMD_MAX,
+};
+
+struct hinic_api_cmd_chain_attr {
+	struct hinic_hwif               *hwif;
+	enum hinic_api_cmd_chain_type   chain_type;
+
+	u32                             num_cells;
+	u16                             cell_size;
+};
+
+struct hinic_api_cmd_status {
+	u64     header;
+	u32     status;
+	u32     rsvd0;
+	u32     rsvd1;
+	u32     rsvd2;
+	u64     rsvd3;
+};
+
+/* HW struct */
+struct hinic_api_cmd_cell {
+	u64 ctrl;
+
+	/* address is 64 bit in HW struct */
+	u64 next_cell_paddr;
+
+	u64 desc;
+
+	/* HW struct */
+	union {
+		struct {
+			u64 hw_cmd_paddr;
+		} write;
+
+		struct {
+			u64 hw_wb_resp_paddr;
+			u64 hw_cmd_paddr;
+		} read;
+	};
+};
+
+struct hinic_api_cmd_cell_ctxt {
+	dma_addr_t                      cell_paddr;
+	struct hinic_api_cmd_cell       *cell_vaddr;
+
+	dma_addr_t                      api_cmd_paddr;
+	u8                              *api_cmd_vaddr;
+};
+
+struct hinic_api_cmd_chain {
+	struct hinic_hwif               *hwif;
+	enum hinic_api_cmd_chain_type   chain_type;
+
+	u32                             num_cells;
+	u16                             cell_size;
+
+	/* HW members in 24 bit format */
+	u32                             prod_idx;
+	u32                             cons_idx;
+
+	struct hinic_api_cmd_cell_ctxt  *cell_ctxt;
+
+	dma_addr_t                      wb_status_paddr;
+	struct hinic_api_cmd_status     *wb_status;
+
+	dma_addr_t                      head_cell_paddr;
+	struct hinic_api_cmd_cell       *head_node;
+	struct hinic_api_cmd_cell       *curr_node;
+};
+
+int hinic_api_cmd_init(struct hinic_api_cmd_chain **chain,
+		       struct hinic_hwif *hwif);
+
+void hinic_api_cmd_free(struct hinic_api_cmd_chain **chain);
+
+#endif
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.c b/drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.c
index 8ae8ed9..f914bc7 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.c
@@ -19,6 +19,7 @@
 
 #include "hinic_hw_if.h"
 #include "hinic_hw_eqs.h"
+#include "hinic_hw_api_cmd.h"
 #include "hinic_hw_mgmt.h"
 #include "hinic_hw_dev.h"
 
@@ -70,9 +71,17 @@ int hinic_pf_to_mgmt_init(struct hinic_pf_to_mgmt *pf_to_mgmt,
 {
 	struct hinic_pfhwdev *pfhwdev = mgmt_to_pfhwdev(pf_to_mgmt);
 	struct hinic_hwdev *hwdev = &pfhwdev->hwdev;
+	struct pci_dev *pdev = hwif->pdev;
+	int err;
 
 	pf_to_mgmt->hwif = hwif;
 
+	err = hinic_api_cmd_init(pf_to_mgmt->cmd_chain, hwif);
+	if (err) {
+		dev_err(&pdev->dev, "Failed to initialize cmd chains\n");
+		return err;
+	}
+
 	hinic_aeq_register_hw_cb(&hwdev->aeqs, HINIC_MSG_FROM_MGMT_CPU,
 				 pf_to_mgmt,
 				 mgmt_msg_aeqe_handler);
@@ -89,4 +98,5 @@ void hinic_pf_to_mgmt_free(struct hinic_pf_to_mgmt *pf_to_mgmt)
 	struct hinic_hwdev *hwdev = &pfhwdev->hwdev;
 
 	hinic_aeq_unregister_hw_cb(&hwdev->aeqs, HINIC_MSG_FROM_MGMT_CPU);
+	hinic_api_cmd_free(pf_to_mgmt->cmd_chain);
 }
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.h b/drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.h
index b4b34b7..dff321c 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.h
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.h
@@ -19,6 +19,7 @@
 #include <linux/types.h>
 
 #include "hinic_hw_if.h"
+#include "hinic_hw_api_cmd.h"
 
 enum hinic_mgmt_msg_type {
 	HINIC_MGMT_MSG_SYNC = 1,
@@ -30,6 +31,8 @@ enum hinic_cfg_cmd {
 
 struct hinic_pf_to_mgmt {
 	struct hinic_hwif               *hwif;
+
+	struct hinic_api_cmd_chain      *cmd_chain[HINIC_API_CMD_MAX];
 };
 
 int hinic_msg_to_mgmt(struct hinic_pf_to_mgmt *pf_to_mgmt,
-- 
1.9.1

[toc] | [prev] | [next] | [standalone]


#1703487 — Re: [PATCH V3 net-next 03/21] net-next/hinic: Initialize api cmd resources

FromDavid Miller <davem@davemloft.net>
Date2017-08-04 00:40 +0200
SubjectRe: [PATCH V3 net-next 03/21] net-next/hinic: Initialize api cmd resources
Message-ID<uawWC-6uI-33@gated-at.bofh.it>
In reply to#1702908
From: Aviad Krawczyk <aviad.krawczyk@huawei.com>
Date: Thu, 3 Aug 2017 17:54:09 +0800

> +static int alloc_cmd_buf(struct hinic_api_cmd_chain *chain,
> +			 struct hinic_api_cmd_cell *cell, int cell_idx)
> +{
> +	struct hinic_hwif *hwif = chain->hwif;
> +	struct pci_dev *pdev = hwif->pdev;
> +	struct hinic_api_cmd_cell_ctxt *cell_ctxt;
> +	dma_addr_t cmd_paddr;
> +	u8 *cmd_vaddr;
> +	int err = 0;

Order local variables from longest to shortest line.

> +static int api_cmd_create_cell(struct hinic_api_cmd_chain *chain,
> +			       int cell_idx,
> +			       struct hinic_api_cmd_cell *pre_node,
> +			       struct hinic_api_cmd_cell **node_vaddr)
> +{
> +	struct hinic_hwif *hwif = chain->hwif;
> +	struct pci_dev *pdev = hwif->pdev;
> +	struct hinic_api_cmd_cell_ctxt *cell_ctxt;
> +	struct hinic_api_cmd_cell *node;
> +	dma_addr_t node_paddr;
> +	int err;

Likewise.
> +static void api_cmd_destroy_cell(struct hinic_api_cmd_chain *chain,
> +				 int cell_idx)
> +{
> +	struct hinic_hwif *hwif = chain->hwif;
> +	struct pci_dev *pdev = hwif->pdev;
> +	struct hinic_api_cmd_cell_ctxt *cell_ctxt;
> +	struct hinic_api_cmd_cell *node;
> +	dma_addr_t node_paddr;
> +	size_t node_size;

Likewise.

etc. etc. etc.

Please audit your entire submission for this problem.

Thanks.

[toc] | [prev] | [next] | [standalone]


#1706741 — Re: [PATCH V3 net-next 03/21] net-next/hinic: Initialize api cmd resources

FromAviad Krawczyk <aviad.krawczyk@huawei.com>
Date2017-08-08 18:30 +0200
SubjectRe: [PATCH V3 net-next 03/21] net-next/hinic: Initialize api cmd resources
Message-ID<ucfyi-2uV-29@gated-at.bofh.it>
In reply to#1703487
Hi David,

Is it a new rule?

We can fix it, but I can look over the Linux Kernel source and there are
a lot of examples that have the same problem. I can see even code that inserted
to 4.10 with the same problem.

Thanks for your time and review,
Aviad

On 8/4/2017 1:35 AM, David Miller wrote:
> From: Aviad Krawczyk <aviad.krawczyk@huawei.com>
> Date: Thu, 3 Aug 2017 17:54:09 +0800
> 
>> +static int alloc_cmd_buf(struct hinic_api_cmd_chain *chain,
>> +			 struct hinic_api_cmd_cell *cell, int cell_idx)
>> +{
>> +	struct hinic_hwif *hwif = chain->hwif;
>> +	struct pci_dev *pdev = hwif->pdev;
>> +	struct hinic_api_cmd_cell_ctxt *cell_ctxt;
>> +	dma_addr_t cmd_paddr;
>> +	u8 *cmd_vaddr;
>> +	int err = 0;
> 
> Order local variables from longest to shortest line.
> 
>> +static int api_cmd_create_cell(struct hinic_api_cmd_chain *chain,
>> +			       int cell_idx,
>> +			       struct hinic_api_cmd_cell *pre_node,
>> +			       struct hinic_api_cmd_cell **node_vaddr)
>> +{
>> +	struct hinic_hwif *hwif = chain->hwif;
>> +	struct pci_dev *pdev = hwif->pdev;
>> +	struct hinic_api_cmd_cell_ctxt *cell_ctxt;
>> +	struct hinic_api_cmd_cell *node;
>> +	dma_addr_t node_paddr;
>> +	int err;
> 
> Likewise.
>> +static void api_cmd_destroy_cell(struct hinic_api_cmd_chain *chain,
>> +				 int cell_idx)
>> +{
>> +	struct hinic_hwif *hwif = chain->hwif;
>> +	struct pci_dev *pdev = hwif->pdev;
>> +	struct hinic_api_cmd_cell_ctxt *cell_ctxt;
>> +	struct hinic_api_cmd_cell *node;
>> +	dma_addr_t node_paddr;
>> +	size_t node_size;
> 
> Likewise.
> 
> etc. etc. etc.
> 
> Please audit your entire submission for this problem.
> 
> Thanks.
> 
> .
> 

[toc] | [prev] | [next] | [standalone]


#1706767 — Re: [PATCH V3 net-next 03/21] net-next/hinic: Initialize api cmd resources

FromDavid Miller <davem@davemloft.net>
Date2017-08-08 19:10 +0200
SubjectRe: [PATCH V3 net-next 03/21] net-next/hinic: Initialize api cmd resources
Message-ID<ucgaZ-2XK-9@gated-at.bofh.it>
In reply to#1706741
From: Aviad Krawczyk <aviad.krawczyk@huawei.com>
Date: Tue, 8 Aug 2017 19:23:32 +0300

> Is it a new rule?

It's at least 10 years old.

> We can fix it, but I can look over the Linux Kernel source and there are
> a lot of examples that have the same problem.

Stop right there.

Just because there is code in the kernel with a problem doesn't mean
you can submit new code with that problem.

[toc] | [prev] | [standalone]


Back to top | Article view | linux.kernel


csiph-web