Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1617440 > unrolled thread

[PATCH] block-mq: set both block queue and hardware queue restart bit for restart

Back to article view | Back to linux.kernel

Contents

  [PATCH] block-mq: set both block queue and hardware queue restart bit for restart Long Li <longli@exchange.microsoft.com> - 2017-04-06 02:20 +0200
    Re: [PATCH] block-mq: set both block queue and hardware queue restart  bit for restart Bart Van Assche <Bart.VanAssche@sandisk.com> - 2017-04-06 02:40 +0200
      RE: [PATCH] block-mq: set both block queue and hardware queue restart  bit for restart Long Li <longli@microsoft.com> - 2017-04-06 05:40 +0200
        Re: [PATCH] block-mq: set both block queue and hardware queue restart  bit for restart Bart Van Assche <Bart.VanAssche@sandisk.com> - 2017-04-06 05:50 +0200
          RE: [PATCH] block-mq: set both block queue and hardware queue restart  bit for restart KY Srinivasan <kys@microsoft.com> - 2017-04-06 06:30 +0200
            RE: [PATCH] block-mq: set both block queue and hardware queue restart  bit for restart Long Li <longli@microsoft.com> - 2017-04-06 07:40 +0200

#1617440 — [PATCH] block-mq: set both block queue and hardware queue restart bit for restart

From: Long Li <longli@microsoft.com>

Under heavy I/O, one hardware queue may be unable to dispatch any I/O to the 
device layer. This poses a problem with restarting this hardware queue on I/O
finish in blk_mq_sched_restart_queues(), becaue there is nothing pending that
will finish in future on this hardware qeueu. This will result in deadlock.

With this patch, we check for all possible stalled hardware queues when I/O
finishes on any hardware queues. This prevents this deadlock.

Signed-off-by: Long Li <longli@microsoft.com>
---
 block/blk-mq-sched.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/block/blk-mq-sched.c b/block/blk-mq-sched.c
index 09af8ff..f7f3d44 100644
--- a/block/blk-mq-sched.c
+++ b/block/blk-mq-sched.c
@@ -202,7 +202,7 @@ void blk_mq_sched_dispatch_requests(struct blk_mq_hw_ctx *hctx)
 	 * needing a restart in that case.
 	 */
 	if (!list_empty(&rq_list)) {
-		blk_mq_sched_mark_restart_hctx(hctx);
+		blk_mq_sched_mark_restart_queue(hctx);
 		did_work = blk_mq_dispatch_rq_list(hctx, &rq_list);
 	} else if (!has_sched_dispatch) {
 		blk_mq_flush_busy_ctxs(hctx, &rq_list);
-- 
2.7.4

[toc] | [next] | [standalone]

#1617443 — Re: [PATCH] block-mq: set both block queue and hardware queue restart bit for restart

On Wed, 2017-04-05 at 17:16 -0700, Long Li wrote:
> Under heavy I/O, one hardware queue may be unable to dispatch any I/O to the 
> device layer. This poses a problem with restarting this hardware queue on I/O
> finish in blk_mq_sched_restart_queues(), becaue there is nothing pending that
> will finish in future on this hardware qeueu. This will result in deadlock.
> 
> With this patch, we check for all possible stalled hardware queues when I/O
> finishes on any hardware queues. This prevents this deadlock.
> 
> Signed-off-by: Long Li <longli@microsoft.com>
> ---
>  block/blk-mq-sched.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/block/blk-mq-sched.c b/block/blk-mq-sched.c
> index 09af8ff..f7f3d44 100644
> --- a/block/blk-mq-sched.c
> +++ b/block/blk-mq-sched.c
> @@ -202,7 +202,7 @@ void blk_mq_sched_dispatch_requests(struct blk_mq_hw_ctx *hctx)
>  	 * needing a restart in that case.
>  	 */
>  	if (!list_empty(&rq_list)) {
> -		blk_mq_sched_mark_restart_hctx(hctx);
> +		blk_mq_sched_mark_restart_queue(hctx);
>  		did_work = blk_mq_dispatch_rq_list(hctx, &rq_list);
>  	} else if (!has_sched_dispatch) {
>  		blk_mq_flush_busy_ctxs(hctx, &rq_list);

Please drop this patch. I'm working on a better solution.

Thanks,

Bart.

[toc] | [prev] | [next] | [standalone]

#1617484 — RE: [PATCH] block-mq: set both block queue and hardware queue restart bit for restart

> -----Original Message-----
> From: Bart Van Assche [mailto:Bart.VanAssche@sandisk.com]
> Sent: Wednesday, April 5, 2017 5:32 PM
> To: linux-kernel@vger.kernel.org; linux-block@vger.kernel.org; Long Li
> <longli@microsoft.com>; axboe@kernel.dk
> Cc: Stephen Hemminger <sthemmin@microsoft.com>; KY Srinivasan
> <kys@microsoft.com>; Long Li <longli@microsoft.com>
> Subject: Re: [PATCH] block-mq: set both block queue and hardware queue
> restart bit for restart
> 
> On Wed, 2017-04-05 at 17:16 -0700, Long Li wrote:
> > Under heavy I/O, one hardware queue may be unable to dispatch any I/O
> > to the device layer. This poses a problem with restarting this
> > hardware queue on I/O finish in blk_mq_sched_restart_queues(), becaue
> > there is nothing pending that will finish in future on this hardware qeueu.
> This will result in deadlock.
> >
> > With this patch, we check for all possible stalled hardware queues
> > when I/O finishes on any hardware queues. This prevents this deadlock.
> >
> > Signed-off-by: Long Li <longli@microsoft.com>
> > ---
> >  block/blk-mq-sched.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/block/blk-mq-sched.c b/block/blk-mq-sched.c index
> > 09af8ff..f7f3d44 100644
> > --- a/block/blk-mq-sched.c
> > +++ b/block/blk-mq-sched.c
> > @@ -202,7 +202,7 @@ void blk_mq_sched_dispatch_requests(struct
> blk_mq_hw_ctx *hctx)
> >  	 * needing a restart in that case.
> >  	 */
> >  	if (!list_empty(&rq_list)) {
> > -		blk_mq_sched_mark_restart_hctx(hctx);
> > +		blk_mq_sched_mark_restart_queue(hctx);
> >  		did_work = blk_mq_dispatch_rq_list(hctx, &rq_list);
> >  	} else if (!has_sched_dispatch) {
> >  		blk_mq_flush_busy_ctxs(hctx, &rq_list);
> 
> Please drop this patch. I'm working on a better solution.

Thank you. Looking forward to your patch.

> 
> Thanks,
> 
> Bart.

[toc] | [prev] | [next] | [standalone]

#1617485 — Re: [PATCH] block-mq: set both block queue and hardware queue restart bit for restart

On Thu, 2017-04-06 at 03:38 +0000, Long Li wrote:
> > -----Original Message-----
> > From: Bart Van Assche [mailto:Bart.VanAssche@sandisk.com]
> > 
> > Please drop this patch. I'm working on a better solution.
> 
> Thank you. Looking forward to your patch.

Hello Long,

It would help if you could share the name of the block or SCSI driver with
which you ran into that lockup and also if you could share the name of the
I/O scheduler used in your test.

Thanks,

Bart.

[toc] | [prev] | [next] | [standalone]

#1617494 — RE: [PATCH] block-mq: set both block queue and hardware queue restart bit for restart

> -----Original Message-----
> From: Bart Van Assche [mailto:Bart.VanAssche@sandisk.com]
> Sent: Wednesday, April 5, 2017 8:46 PM
> To: linux-kernel@vger.kernel.org; linux-block@vger.kernel.org; Long Li
> <longli@microsoft.com>; axboe@kernel.dk
> Cc: Stephen Hemminger <sthemmin@microsoft.com>; KY Srinivasan
> <kys@microsoft.com>
> Subject: Re: [PATCH] block-mq: set both block queue and hardware queue
> restart bit for restart
> 
> On Thu, 2017-04-06 at 03:38 +0000, Long Li wrote:
> > > -----Original Message-----
> > > From: Bart Van Assche [mailto:Bart.VanAssche@sandisk.com]
> > >
> > > Please drop this patch. I'm working on a better solution.
> >
> > Thank you. Looking forward to your patch.
> 
> Hello Long,
> 
> It would help if you could share the name of the block or SCSI driver with
> which you ran into that lockup and also if you could share the name of the
> I/O scheduler used in your test.

The tests that indicated the issue were run Hyper-V. The driver is storvsc_drv.c
The I/O scheduler was I think noop.

K. Y
> 
> Thanks,
> 
> Bart.

[toc] | [prev] | [next] | [standalone]

#1617509 — RE: [PATCH] block-mq: set both block queue and hardware queue restart bit for restart

> -----Original Message-----
> From: KY Srinivasan
> Sent: Wednesday, April 5, 2017 9:21 PM
> To: Bart Van Assche <Bart.VanAssche@sandisk.com>; linux-
> kernel@vger.kernel.org; linux-block@vger.kernel.org; Long Li
> <longli@microsoft.com>; axboe@kernel.dk
> Cc: Stephen Hemminger <sthemmin@microsoft.com>
> Subject: RE: [PATCH] block-mq: set both block queue and hardware queue
> restart bit for restart
> 
> 
> 
> > -----Original Message-----
> > From: Bart Van Assche [mailto:Bart.VanAssche@sandisk.com]
> > Sent: Wednesday, April 5, 2017 8:46 PM
> > To: linux-kernel@vger.kernel.org; linux-block@vger.kernel.org; Long Li
> > <longli@microsoft.com>; axboe@kernel.dk
> > Cc: Stephen Hemminger <sthemmin@microsoft.com>; KY Srinivasan
> > <kys@microsoft.com>
> > Subject: Re: [PATCH] block-mq: set both block queue and hardware queue
> > restart bit for restart
> >
> > On Thu, 2017-04-06 at 03:38 +0000, Long Li wrote:
> > > > -----Original Message-----
> > > > From: Bart Van Assche [mailto:Bart.VanAssche@sandisk.com]
> > > >
> > > > Please drop this patch. I'm working on a better solution.
> > >
> > > Thank you. Looking forward to your patch.
> >
> > Hello Long,
> >
> > It would help if you could share the name of the block or SCSI driver
> > with which you ran into that lockup and also if you could share the
> > name of the I/O scheduler used in your test.
> 
> The tests that indicated the issue were run Hyper-V. The driver is
> storvsc_drv.c The I/O scheduler was I think noop.

Yes, we see I/O hung on scheduler none. Also tried on mq-deadline, same hung with the same cause.

> 
> K. Y
> >
> > Thanks,
> >
> > Bart.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web