Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1328974 > unrolled thread

[PATCH 0/6] net: thunderx: Setting IRQ affinity hints and other optimizations

Back to article view | Back to linux.kernel

Contents

  [PATCH 0/6] net: thunderx: Setting IRQ affinity hints and other optimizations sunil.kovvuri@gmail.com - 2016-02-08 13:10 +0100
    [PATCH 6/6] net: thunderx: Alloc higher order pages when pagesize is small sunil.kovvuri@gmail.com - 2016-02-08 13:10 +0100
    [PATCH 5/6] net: thunderx: bgx: Add log message when setting mac address sunil.kovvuri@gmail.com - 2016-02-08 13:10 +0100
      Re: [PATCH 5/6] net: thunderx: bgx: Add log message when setting mac  address Sergei Shtylyov <sergei.shtylyov@cogentembedded.com> - 2016-02-08 14:40 +0100
        Re: [PATCH 5/6] net: thunderx: bgx: Add log message when setting mac  address Robert Richter <robert.richter@caviumnetworks.com> - 2016-02-08 14:50 +0100
        Re: [PATCH 5/6] net: thunderx: bgx: Add log message when setting mac  address Sergei Shtylyov <sergei.shtylyov@cogentembedded.com> - 2016-02-08 15:00 +0100
        Re: [PATCH 5/6] net: thunderx: bgx: Add log message when setting mac address Bjørn Mork <bjorn@mork.no> - 2016-02-08 15:00 +0100
    [PATCH 3/6] net: thunderx: Assign affinity hints to vf's interrupts sunil.kovvuri@gmail.com - 2016-02-08 13:10 +0100
    [PATCH 2/6] net: thunderx: Use napi_schedule_irqoff() sunil.kovvuri@gmail.com - 2016-02-08 13:10 +0100

#1328974 — [PATCH 0/6] net: thunderx: Setting IRQ affinity hints and other optimizations

From: Sunil Goutham <sgoutham@cavium.com>

This patch series contains changes
- To add support for virtual function's irq affinity hint
- Replace napi_schedule() with napi_schedule_irqoff()
- Reduce page allocation overhead by allocating pages
  of higher order when pagesize is 4KB.
- Add couple of stats which helps in debugging
- Some miscellaneous changes to BGX driver.


David Daney (1):
  net: thunderx: bgx: Use standard firmware node infrastructure.

Robert Richter (1):
  net: thunderx: bgx: Add log message when setting mac address

Sunil Goutham (3):
  net: thunderx: Use napi_schedule_irqoff()
  net: thunderx: Assign affinity hints to vf's interrupts
  net: thunderx: Alloc higher order pages when pagesize is small

Thanneeru Srinivasulu (1):
  net, thunderx: Add TX timeout and RX buffer alloc failure stats.

 drivers/net/ethernet/cavium/thunder/nic.h          |    4 ++
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c    |    2 +
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   |   40 ++++++++++++++++++-
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |    5 +-
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c  |   38 +++++++++++--------
 5 files changed, 67 insertions(+), 22 deletions(-)

[toc] | [next] | [standalone]

#1328975 — [PATCH 6/6] net: thunderx: Alloc higher order pages when pagesize is small

From: Sunil Goutham <sgoutham@cavium.com>

Allocate higher order pages when pagesize is small, this will
reduce number of calls to page allocator and wastage of memory.

Signed-off-by: Sunil Goutham <sgoutham@cavium.com>
---
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
index 50ab6f4..5adb208 100644
--- a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
+++ b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
@@ -78,7 +78,7 @@ static void nicvf_free_q_desc_mem(struct nicvf *nic, struct q_desc_mem *dmem)
 static inline int nicvf_alloc_rcv_buffer(struct nicvf *nic, gfp_t gfp,
 					 u32 buf_len, u64 **rbuf)
 {
-	int order = get_order(buf_len);
+	int order = (PAGE_SIZE <= 4096) ?  PAGE_ALLOC_COSTLY_ORDER : 0;
 
 	/* Check if request can be accomodated in previous allocated page */
 	if (nic->rb_page) {
-- 
1.7.1

[toc] | [prev] | [next] | [standalone]

#1328977 — [PATCH 5/6] net: thunderx: bgx: Add log message when setting mac address

From: Robert Richter <rrichter@cavium.com>

Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: Sunil Goutham <sgoutham@cavium.com>
---
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c |   11 ++++++++---
 1 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/drivers/net/ethernet/cavium/thunder/thunder_bgx.c b/drivers/net/ethernet/cavium/thunder/thunder_bgx.c
index 111835b..cfee496 100644
--- a/drivers/net/ethernet/cavium/thunder/thunder_bgx.c
+++ b/drivers/net/ethernet/cavium/thunder/thunder_bgx.c
@@ -886,7 +886,8 @@ static void bgx_get_qlm_mode(struct bgx *bgx)
 
 #ifdef CONFIG_ACPI
 
-static int acpi_get_mac_address(struct acpi_device *adev, u8 *dst)
+static int acpi_get_mac_address(struct device *dev, struct acpi_device *adev,
+				u8 *dst)
 {
 	u8 mac[ETH_ALEN];
 	int ret;
@@ -897,10 +898,13 @@ static int acpi_get_mac_address(struct acpi_device *adev, u8 *dst)
 		goto out;
 
 	if (!is_valid_ether_addr(mac)) {
+		dev_warn(dev, "MAC address invalid: %pM\n", mac);
 		ret = -EINVAL;
 		goto out;
 	}
 
+	dev_info(dev, "MAC address set to: %pM\n", mac);
+
 	memcpy(dst, mac, ETH_ALEN);
 out:
 	return ret;
@@ -911,14 +915,15 @@ static acpi_status bgx_acpi_register_phy(acpi_handle handle,
 					 u32 lvl, void *context, void **rv)
 {
 	struct bgx *bgx = context;
+	struct device *dev = &bgx->pdev->dev;
 	struct acpi_device *adev;
 
 	if (acpi_bus_get_device(handle, &adev))
 		goto out;
 
-	acpi_get_mac_address(adev, bgx->lmac[bgx->lmac_count].mac);
+	acpi_get_mac_address(dev, adev, bgx->lmac[bgx->lmac_count].mac);
 
-	SET_NETDEV_DEV(&bgx->lmac[bgx->lmac_count].netdev, &bgx->pdev->dev);
+	SET_NETDEV_DEV(&bgx->lmac[bgx->lmac_count].netdev, dev);
 
 	bgx->lmac[bgx->lmac_count].lmacid = bgx->lmac_count;
 out:
-- 
1.7.1

[toc] | [prev] | [next] | [standalone]

#1329056 — Re: [PATCH 5/6] net: thunderx: bgx: Add log message when setting mac address

Hello.

On 2/8/2016 3:07 PM, sunil.kovvuri@gmail.com wrote:

> From: Robert Richter <rrichter@cavium.com>
>
> Signed-off-by: Robert Richter <rrichter@cavium.com>
> Signed-off-by: Sunil Goutham <sgoutham@cavium.com>
> ---
>   drivers/net/ethernet/cavium/thunder/thunder_bgx.c |   11 ++++++++---
>   1 files changed, 8 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/net/ethernet/cavium/thunder/thunder_bgx.c b/drivers/net/ethernet/cavium/thunder/thunder_bgx.c
> index 111835b..cfee496 100644
> --- a/drivers/net/ethernet/cavium/thunder/thunder_bgx.c
> +++ b/drivers/net/ethernet/cavium/thunder/thunder_bgx.c
[...]
> @@ -897,10 +898,13 @@ static int acpi_get_mac_address(struct acpi_device *adev, u8 *dst)
>   		goto out;
>
>   	if (!is_valid_ether_addr(mac)) {
> +		dev_warn(dev, "MAC address invalid: %pM\n", mac);

    dev_er(), maybe?

>   		ret = -EINVAL;
>   		goto out;
>   	}
>
> +	dev_info(dev, "MAC address set to: %pM\n", mac);
> +
>   	memcpy(dst, mac, ETH_ALEN);
>   out:
>   	return ret;
[...]

MBR, Sergei

[toc] | [prev] | [next] | [standalone]

#1329064 — Re: [PATCH 5/6] net: thunderx: bgx: Add log message when setting mac address

On 08.02.16 16:30:37, Sergei Shtylyov wrote:
> >@@ -897,10 +898,13 @@ static int acpi_get_mac_address(struct acpi_device *adev, u8 *dst)
> >  		goto out;
> >
> >  	if (!is_valid_ether_addr(mac)) {
> >+		dev_warn(dev, "MAC address invalid: %pM\n", mac);
> 
>    dev_er(), maybe?

Since the driver may continue, my choice was a warning only.

-Robert

> 
> >  		ret = -EINVAL;
> >  		goto out;
> >  	}
> >
> >+	dev_info(dev, "MAC address set to: %pM\n", mac);
> >+
> >  	memcpy(dst, mac, ETH_ALEN);
> >  out:
> >  	return ret;
> [...]

[toc] | [prev] | [next] | [standalone]

#1329071 — Re: [PATCH 5/6] net: thunderx: bgx: Add log message when setting mac address

On 02/08/2016 04:49 PM, Bjørn Mork wrote:

>>     dev_er(), maybe?
>
> I like that!
>
> It's often hard to know whether to print something or be quiet.
> pr_er(), dev_er(), netdev_er() etc would be the perfect solution to that
> problem.

    :-D
    Sorry for the typo...

> Bjørn

MBR, Sergei

[toc] | [prev] | [next] | [standalone]

#1329078 — Re: [PATCH 5/6] net: thunderx: bgx: Add log message when setting mac address

Sergei Shtylyov <sergei.shtylyov@cogentembedded.com> writes:

>    dev_er(), maybe?

I like that!

It's often hard to know whether to print something or be quiet.
pr_er(), dev_er(), netdev_er() etc would be the perfect solution to that
problem.


Bjørn

[toc] | [prev] | [next] | [standalone]

#1328978 — [PATCH 3/6] net: thunderx: Assign affinity hints to vf's interrupts

From: Sunil Goutham <sgoutham@cavium.com>

This affinity hint can be used by user space irqbalance tool to set
preferred CPU mask for irqs registered by this VF. Irqbalance needs
to be in 'exact' mode to set irq affinity same as indicated by
affinity hint.

Signed-off-by: Sunil Goutham <sgoutham@cavium.com>
---
 drivers/net/ethernet/cavium/thunder/nic.h        |    1 +
 drivers/net/ethernet/cavium/thunder/nicvf_main.c |   37 ++++++++++++++++++++-
 2 files changed, 36 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/cavium/thunder/nic.h b/drivers/net/ethernet/cavium/thunder/nic.h
index 8af363a..00cc915 100644
--- a/drivers/net/ethernet/cavium/thunder/nic.h
+++ b/drivers/net/ethernet/cavium/thunder/nic.h
@@ -309,6 +309,7 @@ struct nicvf {
 	struct msix_entry	msix_entries[NIC_VF_MSIX_VECTORS];
 	char			irq_name[NIC_VF_MSIX_VECTORS][20];
 	bool			irq_allocated[NIC_VF_MSIX_VECTORS];
+	cpumask_var_t		affinity_mask[NIC_VF_MSIX_VECTORS];
 
 	/* VF <-> PF mailbox communication */
 	bool			pf_acked;
diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_main.c b/drivers/net/ethernet/cavium/thunder/nicvf_main.c
index c6f146c..90ce93e 100644
--- a/drivers/net/ethernet/cavium/thunder/nicvf_main.c
+++ b/drivers/net/ethernet/cavium/thunder/nicvf_main.c
@@ -899,6 +899,31 @@ static void nicvf_disable_msix(struct nicvf *nic)
 	}
 }
 
+static void nicvf_set_irq_affinity(struct nicvf *nic)
+{
+	int vec, cpu;
+	int irqnum;
+
+	for (vec = 0; vec < nic->num_vec; vec++) {
+		if (!nic->irq_allocated[vec])
+			continue;
+
+		if (!zalloc_cpumask_var(&nic->affinity_mask[vec], GFP_KERNEL))
+			return;
+		 /* CQ interrupts */
+		if (vec < NICVF_INTR_ID_SQ)
+			/* Leave CPU0 for RBDR and other interrupts */
+			cpu = nicvf_netdev_qidx(nic, vec) + 1;
+		else
+			cpu = 0;
+
+		cpumask_set_cpu(cpumask_local_spread(cpu, nic->node),
+				nic->affinity_mask[vec]);
+		irqnum = nic->msix_entries[vec].vector;
+		irq_set_affinity_hint(irqnum, nic->affinity_mask[vec]);
+	}
+}
+
 static int nicvf_register_interrupts(struct nicvf *nic)
 {
 	int irq, ret = 0;
@@ -944,8 +969,13 @@ static int nicvf_register_interrupts(struct nicvf *nic)
 	ret = request_irq(nic->msix_entries[irq].vector,
 			  nicvf_qs_err_intr_handler,
 			  0, nic->irq_name[irq], nic);
-	if (!ret)
-		nic->irq_allocated[irq] = true;
+	if (ret)
+		goto err;
+
+	nic->irq_allocated[irq] = true;
+
+	/* Set IRQ affinities */
+	nicvf_set_irq_affinity(nic);
 
 err:
 	if (ret)
@@ -963,6 +993,9 @@ static void nicvf_unregister_interrupts(struct nicvf *nic)
 		if (!nic->irq_allocated[irq])
 			continue;
 
+		irq_set_affinity_hint(nic->msix_entries[irq].vector, NULL);
+		free_cpumask_var(nic->affinity_mask[irq]);
+
 		if (irq < NICVF_INTR_ID_SQ)
 			free_irq(nic->msix_entries[irq].vector, nic->napi[irq]);
 		else
-- 
1.7.1

[toc] | [prev] | [next] | [standalone]

#1328983 — [PATCH 2/6] net: thunderx: Use napi_schedule_irqoff()

From: Sunil Goutham <sgoutham@cavium.com>

napi_schedule is being called from hard irq context, hence
switch to napi_schedule_irqoff which avoids unneeded call
to local_irq_save and local_irq_restore.

Signed-off-by: Sunil Goutham <sgoutham@cavium.com>
---
 drivers/net/ethernet/cavium/thunder/nicvf_main.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_main.c b/drivers/net/ethernet/cavium/thunder/nicvf_main.c
index 95db6b7..c6f146c 100644
--- a/drivers/net/ethernet/cavium/thunder/nicvf_main.c
+++ b/drivers/net/ethernet/cavium/thunder/nicvf_main.c
@@ -828,7 +828,7 @@ static irqreturn_t nicvf_intr_handler(int irq, void *cq_irq)
 	nicvf_disable_intr(nic, NICVF_INTR_CQ, qidx);
 
 	/* Schedule NAPI */
-	napi_schedule(&cq_poll->napi);
+	napi_schedule_irqoff(&cq_poll->napi);
 
 	/* Clear interrupt */
 	nicvf_clear_intr(nic, NICVF_INTR_CQ, qidx);
-- 
1.7.1

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web