Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1642129 > unrolled thread

RE: [PATCH] scsi: zero per-cmd driver data for each MQ I/O

Back to article view | Back to linux.kernel

Contents

  RE: [PATCH] scsi: zero per-cmd driver data for each MQ I/O Long Li <longli@microsoft.com> - 2017-05-16 01:40 +0200
    Re: [PATCH] scsi: zero per-cmd driver data for each MQ I/O Bart Van Assche <Bart.VanAssche@sandisk.com> - 2017-05-16 02:10 +0200
      RE: [PATCH] scsi: zero per-cmd driver data for each MQ I/O Long Li <longli@microsoft.com> - 2017-05-16 19:40 +0200
        Re: [PATCH] scsi: zero per-cmd driver data for each MQ I/O Bart Van Assche <Bart.VanAssche@sandisk.com> - 2017-05-18 01:10 +0200
          Re: [PATCH] scsi: zero per-cmd driver data for each MQ I/O Christoph Hellwig <hch@infradead.org> - 2017-05-18 09:00 +0200
            RE: [PATCH] scsi: zero per-cmd driver data for each MQ I/O KY Srinivasan <kys@microsoft.com> - 2017-05-18 15:30 +0200
            Re: [PATCH] scsi: zero per-cmd driver data for each MQ I/O Bart Van Assche <Bart.VanAssche@sandisk.com> - 2017-05-18 18:00 +0200
              RE: [PATCH] scsi: zero per-cmd driver data for each MQ I/O Long Li <longli@microsoft.com> - 2017-05-18 18:50 +0200

#1642129 — RE: [PATCH] scsi: zero per-cmd driver data for each MQ I/O

> -----Original Message-----
> From: Bart Van Assche [mailto:Bart.VanAssche@sandisk.com]
> Sent: Monday, May 15, 2017 4:03 PM
> To: jejb@linux.vnet.ibm.com; linux-scsi@vger.kernel.org; linux-
> kernel@vger.kernel.org; Long Li <longli@microsoft.com>;
> martin.petersen@oracle.com
> Cc: Long Li <longli@microsoft.com>
> Subject: Re: [Possible Phish Fraud][PATCH] scsi: zero per-cmd driver data for
> each MQ I/O
> 
> On Wed, 2017-05-10 at 14:07 -0700, Long Li wrote:
> > From: Long Li <longli@microsoft.com>
> >
> > Lower layer driver may not initialize private data before use. Zero
> > them out to prevent use of stale data.
> >
> > Signed-off-by: Long Li <longli@microsoft.com>
> > ---
> >  drivers/scsi/scsi_lib.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c index
> > 19125d7..a821593 100644
> > --- a/drivers/scsi/scsi_lib.c
> > +++ b/drivers/scsi/scsi_lib.c
> > @@ -1850,7 +1850,7 @@ static int scsi_mq_prep_fn(struct request *req)
> >
> >  	/* zero out the cmd, except for the embedded scsi_request */
> >  	memset((char *)cmd + sizeof(cmd->req), 0,
> > -		sizeof(*cmd) - sizeof(cmd->req));
> > +		sizeof(*cmd) - sizeof(cmd->req) + shost->hostt->cmd_size);
> >
> >  	req->special = cmd;
> 
> Hello Long,
> 
> Sorry but this patch looks wrong to me. Since scsi_mq_prep_fn() is called
> after scsi_req_init(), erasing struct scsi_request from scsi_mq_prep_fn() will
> erase the values that were set by scsi_req_init(). That includes information
> like the pointer to the SCSI CDB and the CDB itself. See e.g.
> scsi_execute().
> 
> Did you come up with this patch after source reading or did you come up with
> this patch while chasing a bug?

Thanks for looking! Yes this is for chasing a bug. 

Actually scsi_mq_prep_fn() doesn't touch cmd->req (which is the struct scsi_request mentioned in your email).

With the patch, we also zero the private data used by lower layer driver, in addition to the private data in scsi_cmnd.

> 
> Thanks,
> 
> Bart.

[toc] | [next] | [standalone]

#1642135

On Mon, 2017-05-15 at 23:32 +0000, Long Li wrote:
> Thanks for looking! Yes this is for chasing a bug. 
> 
> With the patch, we also zero the private data used by lower layer driver, in
> addition to the private data in scsi_cmnd.

Hello Long,

What bug did you encounter, with which combination of ULP (sd?) and LLD SCSI
driver(s) and for which request type (REQ_OP_*)? You will have to mention that
information in the patch description anyway if you want your patch to get
accepted.

If the bug that you encountered only occurs with a single LLD, would it be
possible to implement a fix by modifying the LLD instead of the SCSI core?

Thanks,

Bart.

[toc] | [prev] | [next] | [standalone]

#1642688

> -----Original Message-----
> From: Bart Van Assche [mailto:Bart.VanAssche@sandisk.com]
> Sent: Monday, May 15, 2017 5:01 PM
> To: jejb@linux.vnet.ibm.com; linux-scsi@vger.kernel.org; linux-
> kernel@vger.kernel.org; Long Li <longli@microsoft.com>;
> martin.petersen@oracle.com
> Cc: Stephen Hemminger <sthemmin@microsoft.com>; KY Srinivasan
> <kys@microsoft.com>
> Subject: Re: [PATCH] scsi: zero per-cmd driver data for each MQ I/O
> 
> On Mon, 2017-05-15 at 23:32 +0000, Long Li wrote:
> > Thanks for looking! Yes this is for chasing a bug.
> >
> > With the patch, we also zero the private data used by lower layer
> > driver, in addition to the private data in scsi_cmnd.
> 
> Hello Long,
> 
> What bug did you encounter, with which combination of ULP (sd?) and LLD SCSI
> driver(s) and for which request type (REQ_OP_*)? You will have to mention
> that information in the patch description anyway if you want your patch to get
> accepted.
> 
> If the bug that you encountered only occurs with a single LLD, would it be
> possible to implement a fix by modifying the LLD instead of the SCSI core?

The bug I encounter is that in hv_storvsc (a LLD), sometime we are getting stale data in the private driver data memory allocated by SCSI. As a LLD, we expect the memory allocated by SCSI to be zeroed. If not we may send unexpected commands to the device.

A little background on private data: In LLD's scsi_host_template, the driver may optionally ask SCSI to allocate its private driver memory for each command, by specifying cmd_size. This memory is allocated at the end of scsi_cmnd by SCSI. Later when SCSI queues a command, the LLD can use scsi_cmd_priv to get its private data.

hv_storvsc doesn't clear its private data before use. I'm not sure about other LLD drivers. Although it's possible to fix it in LLD not SCSI core, I think that is not the ideal place to do it. Whoever is allocating the SCSI command should also zero it.

There is a similar patch that fixed a similar issue for non-MQ case:
commit ee5242360424b9b967454e9183767323d10cf985

I'm sorry I should have put more details in the patch. 

> 
> Thanks,
> 
> Bart.

[toc] | [prev] | [next] | [standalone]

#1643687

On Tue, 2017-05-16 at 17:31 +0000, Long Li wrote:
> > -----Original Message-----
> > From: Bart Van Assche [mailto:Bart.VanAssche@sandisk.com]
> > Sent: Monday, May 15, 2017 5:01 PM
> > To: jejb@linux.vnet.ibm.com; linux-scsi@vger.kernel.org; linux-
> > kernel@vger.kernel.org; Long Li <longli@microsoft.com>;
> > martin.petersen@oracle.com
> > Cc: Stephen Hemminger <sthemmin@microsoft.com>; KY Srinivasan
> > <kys@microsoft.com>
> > Subject: Re: [PATCH] scsi: zero per-cmd driver data for each MQ I/O
> > 
> > On Mon, 2017-05-15 at 23:32 +0000, Long Li wrote:
> > > Thanks for looking! Yes this is for chasing a bug.
> > > 
> > > With the patch, we also zero the private data used by lower layer
> > > driver, in addition to the private data in scsi_cmnd.
> > 
> > Hello Long,
> > 
> > What bug did you encounter, with which combination of ULP (sd?) and LLD SCSI
> > driver(s) and for which request type (REQ_OP_*)? You will have to mention
> > that information in the patch description anyway if you want your patch to get
> > accepted.
> > 
> > If the bug that you encountered only occurs with a single LLD, would it be
> > possible to implement a fix by modifying the LLD instead of the SCSI core?
> 
> The bug I encounter is that in hv_storvsc (a LLD), sometime we are getting stale data in the private driver data memory allocated by SCSI. As a LLD, we expect the memory allocated by SCSI to be zeroed. If not we may send unexpected commands to the device.
> 
> A little background on private data: In LLD's scsi_host_template, the driver may optionally ask SCSI to allocate its private driver memory for each command, by specifying cmd_size. This memory is allocated at the end of scsi_cmnd by SCSI. Later when SCSI queues a command, the LLD can use scsi_cmd_priv to get its private data.
> 
> hv_storvsc doesn't clear its private data before use. I'm not sure about other LLD drivers. Although it's possible to fix it in LLD not SCSI core, I think that is not the ideal place to do it. Whoever is allocating the SCSI command should also zero it.
> 
> There is a similar patch that fixed a similar issue for non-MQ case:
> commit ee5242360424b9b967454e9183767323d10cf985
> 
> I'm sorry I should have put more details in the patch. 

Hello Long,

Thank you for the feedback. I'm working on a patch series that merges the scsi-sq
and scsi-mq code paths for command initialization and that should fix the bug you
encountered.

Bart.

[toc] | [prev] | [next] | [standalone]

#1643880

On Wed, May 17, 2017 at 11:05:18PM +0000, Bart Van Assche wrote:
> Thank you for the feedback. I'm working on a patch series that merges the scsi-sq
> and scsi-mq code paths for command initialization and that should fix the bug you
> encountered.

While that sounds great (I tried it a while ago but gave up due to
priorities) I think we should merge this patch as-is and backport it
to stable for now and rebase your series on top of it.

[toc] | [prev] | [next] | [standalone]

#1644516

> -----Original Message-----
> From: Christoph Hellwig [mailto:hch@infradead.org]
> Sent: Wednesday, May 17, 2017 11:55 PM
> To: Bart Van Assche <Bart.VanAssche@sandisk.com>
> Cc: jejb@linux.vnet.ibm.com; linux-scsi@vger.kernel.org; linux-
> kernel@vger.kernel.org; Long Li <longli@microsoft.com>;
> martin.petersen@oracle.com; Stephen Hemminger
> <sthemmin@microsoft.com>; KY Srinivasan <kys@microsoft.com>
> Subject: Re: [PATCH] scsi: zero per-cmd driver data for each MQ I/O
> 
> On Wed, May 17, 2017 at 11:05:18PM +0000, Bart Van Assche wrote:
> > Thank you for the feedback. I'm working on a patch series that merges the
> scsi-sq
> > and scsi-mq code paths for command initialization and that should fix the
> bug you
> > encountered.
> 
> While that sounds great (I tried it a while ago but gave up due to
> priorities) I think we should merge this patch as-is and backport it
> to stable for now and rebase your series on top of it.
Thanks Christoph.

K. Y

[toc] | [prev] | [next] | [standalone]

#1644713

On Wed, 2017-05-17 at 23:54 -0700, Christoph Hellwig wrote:
> On Wed, May 17, 2017 at 11:05:18PM +0000, Bart Van Assche wrote:
> > Thank you for the feedback. I'm working on a patch series that merges the scsi-sq
> > and scsi-mq code paths for command initialization and that should fix the bug you
> > encountered.
> 
> While that sounds great (I tried it a while ago but gave up due to
> priorities) I think we should merge this patch as-is and backport it
> to stable for now and rebase your series on top of it.

Hello Christoph,

I will rebase my patch series on top of Long Li's patch. Long, please
repost your patch with a more detailed description and with the Reviewed-by
and Cc: stable tags added.

Thanks,

Bart.

[toc] | [prev] | [next] | [standalone]

#1644749

> -----Original Message-----
> From: Bart Van Assche [mailto:Bart.VanAssche@sandisk.com]
> Sent: Thursday, May 18, 2017 8:52 AM
> To: hch@infradead.org
> Cc: jejb@linux.vnet.ibm.com; linux-scsi@vger.kernel.org; linux-
> kernel@vger.kernel.org; Stephen Hemminger <sthemmin@microsoft.com>; KY
> Srinivasan <kys@microsoft.com>; Long Li <longli@microsoft.com>;
> martin.petersen@oracle.com
> Subject: Re: [PATCH] scsi: zero per-cmd driver data for each MQ I/O
> 
> On Wed, 2017-05-17 at 23:54 -0700, Christoph Hellwig wrote:
> > On Wed, May 17, 2017 at 11:05:18PM +0000, Bart Van Assche wrote:
> > > Thank you for the feedback. I'm working on a patch series that
> > > merges the scsi-sq and scsi-mq code paths for command initialization
> > > and that should fix the bug you encountered.
> >
> > While that sounds great (I tried it a while ago but gave up due to
> > priorities) I think we should merge this patch as-is and backport it
> > to stable for now and rebase your series on top of it.
> 
> Hello Christoph,
> 
> I will rebase my patch series on top of Long Li's patch. Long, please repost your
> patch with a more detailed description and with the Reviewed-by and Cc:
> stable tags added.

Will do that.

> 
> Thanks,
> 
> Bart.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web