Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1256864 > unrolled thread

[PATCH] IB/sa: replace GFP_KERNEL with GFP_ATOMIC

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  [PATCH] IB/sa: replace GFP_KERNEL with GFP_ATOMIC Saurabh Sengar <saurabh.truth@gmail.com> - 2015-10-27 16:50 +0100
    Re: [PATCH] IB/sa: replace GFP_KERNEL with GFP_ATOMIC "ira.weiny" <ira.weiny@intel.com> - 2015-10-27 19:20 +0100
      Re: [PATCH] IB/sa: replace GFP_KERNEL with GFP_ATOMIC Jason Gunthorpe <jgunthorpe@obsidianresearch.com> - 2015-10-27 19:20 +0100
        Re: [PATCH] IB/sa: replace GFP_KERNEL with GFP_ATOMIC "ira.weiny" <ira.weiny@intel.com> - 2015-10-27 19:30 +0100
        RE: [PATCH] IB/sa: replace GFP_KERNEL with GFP_ATOMIC "Wan, Kaike" <kaike.wan@intel.com> - 2015-10-27 20:00 +0100
          Re: [PATCH] IB/sa: replace GFP_KERNEL with GFP_ATOMIC Jason Gunthorpe <jgunthorpe@obsidianresearch.com> - 2015-10-27 21:10 +0100
            RE: [PATCH] IB/sa: replace GFP_KERNEL with GFP_ATOMIC "Weiny, Ira" <ira.weiny@intel.com> - 2015-10-28 05:40 +0100

#1256864 — [PATCH] IB/sa: replace GFP_KERNEL with GFP_ATOMIC

replace GFP_KERNEL with GFP_ATOMIC, as code while holding a spinlock
should be atomic
GFP_KERNEL may sleep and can cause deadlock, where as GFP_ATOMIC may
fail but certainly avoids deadlock

Signed-off-by: Saurabh Sengar <saurabh.truth@gmail.com>
---
 drivers/infiniband/core/sa_query.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c
index 8c014b3..cd1f911 100644
--- a/drivers/infiniband/core/sa_query.c
+++ b/drivers/infiniband/core/sa_query.c
@@ -526,7 +526,7 @@ static int ib_nl_send_msg(struct ib_sa_query *query)
 	if (len <= 0)
 		return -EMSGSIZE;
 
-	skb = nlmsg_new(len, GFP_KERNEL);
+	skb = nlmsg_new(len, GFP_ATOMIC);
 	if (!skb)
 		return -ENOMEM;
 
@@ -544,7 +544,7 @@ static int ib_nl_send_msg(struct ib_sa_query *query)
 	/* Repair the nlmsg header length */
 	nlmsg_end(skb, nlh);
 
-	ret = ibnl_multicast(skb, nlh, RDMA_NL_GROUP_LS, GFP_KERNEL);
+	ret = ibnl_multicast(skb, nlh, RDMA_NL_GROUP_LS, GFP_ATOMIC);
 	if (!ret)
 		ret = len;
 	else
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1257106

On Tue, Oct 27, 2015 at 09:17:40PM +0530, Saurabh Sengar wrote:
> replace GFP_KERNEL with GFP_ATOMIC, as code while holding a spinlock
> should be atomic
> GFP_KERNEL may sleep and can cause deadlock, where as GFP_ATOMIC may
> fail but certainly avoids deadlock

Great catch.  Thanks!

However, gfp_t is passed to send_mad and we should pass that down and use it.

Compile tested only, suggestion below,
Ira


14:09:12 > git di
diff --git a/drivers/infiniband/core/sa_query.c
b/drivers/infiniband/core/sa_query.c
index 8c014b33d8e0..54d454042b28 100644
--- a/drivers/infiniband/core/sa_query.c
+++ b/drivers/infiniband/core/sa_query.c
@@ -512,7 +512,7 @@ static int ib_nl_get_path_rec_attrs_len(ib_sa_comp_mask comp_mask)
        return len;
 }
 
-static int ib_nl_send_msg(struct ib_sa_query *query)
+static int ib_nl_send_msg(struct ib_sa_query *query, gfp_t gfp_mask)
 {
        struct sk_buff *skb = NULL;
        struct nlmsghdr *nlh;
@@ -526,7 +526,7 @@ static int ib_nl_send_msg(struct ib_sa_query *query)
        if (len <= 0)
                return -EMSGSIZE;
 
-       skb = nlmsg_new(len, GFP_KERNEL);
+       skb = nlmsg_new(len, gfp_mask);
        if (!skb)
                return -ENOMEM;
 
@@ -544,7 +544,7 @@ static int ib_nl_send_msg(struct ib_sa_query *query)
        /* Repair the nlmsg header length */
        nlmsg_end(skb, nlh);
 
-       ret = ibnl_multicast(skb, nlh, RDMA_NL_GROUP_LS, GFP_KERNEL);
+       ret = ibnl_multicast(skb, nlh, RDMA_NL_GROUP_LS, gfp_mask);
        if (!ret)
                ret = len;
        else
@@ -553,7 +553,7 @@ static int ib_nl_send_msg(struct ib_sa_query *query)
        return ret;
 }
 
-static int ib_nl_make_request(struct ib_sa_query *query)
+static int ib_nl_make_request(struct ib_sa_query *query, gfp_t gfp_mask)
 {
        unsigned long flags;
        unsigned long delay;
@@ -563,7 +563,7 @@ static int ib_nl_make_request(struct ib_sa_query *query)
        query->seq = (u32)atomic_inc_return(&ib_nl_sa_request_seq);
 
        spin_lock_irqsave(&ib_nl_request_lock, flags);
-       ret = ib_nl_send_msg(query);
+       ret = ib_nl_send_msg(query, gfp_mask);
        if (ret <= 0) {
                ret = -EIO;
                goto request_out;
@@ -1105,7 +1105,7 @@ static int send_mad(struct ib_sa_query *query, int timeout_ms, gfp_t gfp_mask)
 
        if (query->flags & IB_SA_ENABLE_LOCAL_SERVICE) {
                if (!ibnl_chk_listeners(RDMA_NL_GROUP_LS)) {
-                       if (!ib_nl_make_request(query))
+                       if (!ib_nl_make_request(query, gfp_mask))
                                return id;
                }
                ib_sa_disable_local_svc(query);

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1257112

On Tue, Oct 27, 2015 at 02:12:36PM -0400, ira.weiny wrote:
> On Tue, Oct 27, 2015 at 09:17:40PM +0530, Saurabh Sengar wrote:
> > replace GFP_KERNEL with GFP_ATOMIC, as code while holding a spinlock
> > should be atomic
> > GFP_KERNEL may sleep and can cause deadlock, where as GFP_ATOMIC may
> > fail but certainly avoids deadlock
> 
> Great catch.  Thanks!
> 
> However, gfp_t is passed to send_mad and we should pass that down and use it.

>         spin_lock_irqsave(&ib_nl_request_lock, flags);
> -       ret = ib_nl_send_msg(query);
> +       ret = ib_nl_send_msg(query, gfp_mask);

A spin lock is guarenteed held around ib_nl_send_msg, so it's
allocations have to be atomic, can't use gfp_mask here..

I do wonder if it is a good idea to call ib_nl_send_msg with a spinlock
held though.. Would be nice to see that go away.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1257115

On Tue, Oct 27, 2015 at 12:16:52PM -0600, Jason Gunthorpe wrote:
> On Tue, Oct 27, 2015 at 02:12:36PM -0400, ira.weiny wrote:
> > On Tue, Oct 27, 2015 at 09:17:40PM +0530, Saurabh Sengar wrote:
> > > replace GFP_KERNEL with GFP_ATOMIC, as code while holding a spinlock
> > > should be atomic
> > > GFP_KERNEL may sleep and can cause deadlock, where as GFP_ATOMIC may
> > > fail but certainly avoids deadlock
> > 
> > Great catch.  Thanks!
> > 
> > However, gfp_t is passed to send_mad and we should pass that down and use it.
> 
> >         spin_lock_irqsave(&ib_nl_request_lock, flags);
> > -       ret = ib_nl_send_msg(query);
> > +       ret = ib_nl_send_msg(query, gfp_mask);
> 
> A spin lock is guarenteed held around ib_nl_send_msg, so it's
> allocations have to be atomic, can't use gfp_mask here..
> 
> I do wonder if it is a good idea to call ib_nl_send_msg with a spinlock
> held though.. Would be nice to see that go away.

Ah, yea my bad.

Ira

> 
> Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1257129

> -----Original Message-----
> From: Jason Gunthorpe [mailto:jgunthorpe@obsidianresearch.com]
> Sent: Tuesday, October 27, 2015 2:17 PM
> To: Weiny, Ira
> Cc: Saurabh Sengar; dledford@redhat.com; Hefty, Sean;
> hal.rosenstock@gmail.com; yun.wang@profitbricks.com; Wan, Kaike; linux-
> rdma@vger.kernel.org; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH] IB/sa: replace GFP_KERNEL with GFP_ATOMIC
> 
> On Tue, Oct 27, 2015 at 02:12:36PM -0400, ira.weiny wrote:
> > On Tue, Oct 27, 2015 at 09:17:40PM +0530, Saurabh Sengar wrote:
> > > replace GFP_KERNEL with GFP_ATOMIC, as code while holding a spinlock
> > > should be atomic GFP_KERNEL may sleep and can cause deadlock, where
> > > as GFP_ATOMIC may fail but certainly avoids deadlock
> >
> > Great catch.  Thanks!
> >
> > However, gfp_t is passed to send_mad and we should pass that down and
> use it.
> 
> >         spin_lock_irqsave(&ib_nl_request_lock, flags);
> > -       ret = ib_nl_send_msg(query);
> > +       ret = ib_nl_send_msg(query, gfp_mask);
> 
> A spin lock is guarenteed held around ib_nl_send_msg, so it's allocations
> have to be atomic, can't use gfp_mask here..
> 
> I do wonder if it is a good idea to call ib_nl_send_msg with a spinlock held
> though.. Would be nice to see that go away.

We have to hold the lock to protect against a race condition that a quick response will try to free the request from the ib_nl_request_list before we even put it on the list.

> 
> Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1257172

On Tue, Oct 27, 2015 at 06:56:50PM +0000, Wan, Kaike wrote:
 
> > I do wonder if it is a good idea to call ib_nl_send_msg with a spinlock held
> > though.. Would be nice to see that go away.
> 
> We have to hold the lock to protect against a race condition that a
> quick response will try to free the request from the
> ib_nl_request_list before we even put it on the list.

Put is on the list first? Use a kref? Doesn't look like a big deal to
clean this up.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1257651

> 
> On Tue, Oct 27, 2015 at 06:56:50PM +0000, Wan, Kaike wrote:
> 
> > > I do wonder if it is a good idea to call ib_nl_send_msg with a
> > > spinlock held though.. Would be nice to see that go away.
> >
> > We have to hold the lock to protect against a race condition that a
> > quick response will try to free the request from the
> > ib_nl_request_list before we even put it on the list.
> 
> Put is on the list first? Use a kref? Doesn't look like a big deal to clean this up.
> 
> Jason

Not sure what "Put is on the list first?" means.  I think it is valid to build the request, 
if success, add to the list, then send it.  That would solve the problem you mention 
above.  Was that what you hand in mind, Jason?

I don't have time to work on this right now, not sure about Kaike.  Until we can remove 
the spinlock the current proposed patch should be applied in the interim.  Sorry for the
noise before.

Reviewed-By: Ira Weiny <ira.weiny@intel.com>

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web