Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1275007
| From | Mark Salter <msalter@redhat.com> |
|---|---|
| Newsgroups | linux.kernel |
| Subject | Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! |
| Date | 2015-11-23 03:00 +0100 |
| Message-ID | <qxOAa-3v8-1@gated-at.bofh.it> (permalink) |
| References | (4 earlier) <qwSPw-76t-19@gated-at.bofh.it> <qxeGm-4EV-5@gated-at.bofh.it> <qxjG2-7Xx-15@gated-at.bofh.it> <qxMf0-28o-5@gated-at.bofh.it> <qxNkK-2MK-7@gated-at.bofh.it> |
| Organization | Red Hat, Inc |
On Mon, 2015-11-23 at 08:36 +0800, Ming Lei wrote:
> On Mon, Nov 23, 2015 at 7:20 AM, Mark Salter <msalter@redhat.com> wrote:
> > On Sun, 2015-11-22 at 00:56 +0800, Ming Lei wrote:
> > > On Sat, 21 Nov 2015 12:30:14 +0100
> > > Laurent Dufour <ldufour@linux.vnet.ibm.com> wrote:
> > >
> > > > On 20/11/2015 13:10, Michael Ellerman wrote:
> > > > > On Thu, 2015-11-19 at 00:23 -0800, Christoph Hellwig wrote:
> > > > >
> > > > > > It's pretty much guaranteed a block layer bug, most likely in the
> > > > > > merge bios to request infrastucture where we don't obey the merging
> > > > > > limits properly.
> > > > > >
> > > > > > Does either of you have a known good and first known bad kernel?
> > > > >
> > > > > Not me, I've only hit it one or two times. All I can say is I have hit it in
> > > > > 4.4-rc1.
> > > > >
> > > > > Laurent, can you narrow it down at all?
> > > >
> > > > It seems that the panic is triggered by the commit bdced438acd8 ("block:
> > > > setup bi_phys_segments after splitting") which has been pulled by the
> > > > merge d9734e0d1ccf ("Merge branch 'for-4.4/core' of
> > > > git://git.kernel.dk/linux-block").
> > > >
> > > > My system is panicing promptly when running a kernel built at
> > > > d9734e0d1ccf, while reverting the commit bdced438acd8, it can run hours
> > > > without panicing.
> > > >
> > > > This being said, I can't explain what's going wrong.
> > > >
> > > > May Ming shed some light here ?
> > >
> > > Laurent, looks there is one bug in blk_bio_segment_split(), would you
> > > mind testing the following patch to see if it fixes your issue?
> > >
> > > ---
> > > From 6fc701231dcc000bc8bc4b9105583380d9aa31f4 Mon Sep 17 00:00:00 2001
> > > From: Ming Lei <ming.lei@canonical.com>
> > > Date: Sun, 22 Nov 2015 00:47:13 +0800
> > > Subject: [PATCH] block: fix segment split
> > >
> > > Inside blk_bio_segment_split(), previous bvec pointer('bvprvp')
> > > always points to the iterator local variable, which is obviously
> > > wrong, so fix it by pointing to the local variable of 'bvprv'.
> > >
> > > Signed-off-by: Ming Lei <ming.lei@canonical.com>
> > > ---
> > > block/blk-merge.c | 4 ++--
> > > 1 file changed, 2 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/block/blk-merge.c b/block/blk-merge.c
> > > index de5716d8..f2efe8a 100644
> > > --- a/block/blk-merge.c
> > > +++ b/block/blk-merge.c
> > > @@ -98,7 +98,7 @@ static struct bio *blk_bio_segment_split(struct request_queue *q,
> > >
> > > seg_size += bv.bv_len;
> > > bvprv = bv;
> > > - bvprvp = &bv;
> > > + bvprvp = &bvprv;
> > > sectors += bv.bv_len >> 9;
> > > continue;
> > > }
> > > @@ -108,7 +108,7 @@ new_segment:
> > >
> > > nsegs++;
> > > bvprv = bv;
> > > - bvprvp = &bv;
> > > + bvprvp = &bvprv;
> > > seg_size = bv.bv_len;
> > > sectors += bv.bv_len >> 9;
> > > }
> >
> > I'm still hitting the BUG even with this patch applied on top of 4.4-rc1.
>
> OK, looks there are still other bugs, care to share us how to reproduce
> it on arm64?
>
> thanks,
> Ming
Unfortunately, the best reproducer I have is to boot the platform. I have seen the
BUG a few times post-boot, but I don't have a consistant reproducer. I am using
upstream 4.4-rc1 with this config:
http://people.redhat.com/msalter/fh_defconfig
With 4.4-rc1 on an APM Mustang platform, I see the BUG about once every 6-7 boots.
On an AMD Seattle platform, about every 9 boots.
I have a script that loops through an ssh command to reboot the platform under test.
I manually install test kernels and then run the script and wait for failure. While
debugging, I have tried more minimal configs with which I have been unable to
reproduce the problem even after several hours of reboots. With the above mentioned
fh_defconfig, I have been able to get a failure within 20 or so boots with most
kernel builds but at certain kernel commits, the failure has taken a longer time to
reproduce.
From my POV, I can't say which commit causes the problem. So far, I have not been
able to reproduce at all before commit d9734e0d1ccf but I am currently trying to
reproduce with commit 0d51ce9ca1116 (one merge earlier than d9734e0d1ccf).
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
kernel BUG at drivers/scsi/scsi_lib.c:1096! Michael Ellerman <mpe@ellerman.id.au> - 2015-11-18 10:20 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Mark Salter <msalter@redhat.com> - 2015-11-18 15:10 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Michael Ellerman <mpe@ellerman.id.au> - 2015-11-19 02:10 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Christoph Hellwig <hch@infradead.org> - 2015-11-19 09:30 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Hannes Reinecke <hare@suse.de> - 2015-11-19 16:40 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Ewan Milne <emilne@redhat.com> - 2015-11-20 15:40 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Hannes Reinecke <hare@suse.de> - 2015-11-20 16:10 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Ewan Milne <emilne@redhat.com> - 2015-11-20 16:30 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Hannes Reinecke <hare@suse.de> - 2015-11-23 08:00 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Hannes Reinecke <hare@suse.de> - 2015-11-25 10:10 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Jens Axboe <axboe@fb.com> - 2015-11-25 19:00 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Hannes Reinecke <hare@suse.de> - 2015-11-25 20:20 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Jens Axboe <axboe@fb.com> - 2015-11-25 20:30 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Mike Snitzer <snitzer@redhat.com> - 2015-11-25 21:30 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Mike Snitzer <snitzer@redhat.com> - 2015-11-25 22:30 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Mike Snitzer <snitzer@redhat.com> - 2015-11-25 19:10 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Hannes Reinecke <hare@suse.de> - 2015-11-25 20:10 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Michael Ellerman <mpe@ellerman.id.au> - 2015-11-20 13:20 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Laurent Dufour <ldufour@linux.vnet.ibm.com> - 2015-11-20 14:00 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Mark Salter <msalter@redhat.com> - 2015-11-20 14:40 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Laurent Dufour <ldufour@linux.vnet.ibm.com> - 2015-11-21 12:40 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Ming Lei <ming.lei@canonical.com> - 2015-11-21 18:00 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Mark Salter <msalter@redhat.com> - 2015-11-23 00:30 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Ming Lei <ming.lei@canonical.com> - 2015-11-23 01:40 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Mark Salter <msalter@redhat.com> - 2015-11-23 03:00 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Ming Lei <ming.lei@canonical.com> - 2015-11-23 03:50 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Ming Lei <tom.leiming@gmail.com> - 2015-11-23 16:30 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Alan Ott <alan@softiron.co.uk> - 2015-11-24 20:40 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Laurent Dufour <ldufour@linux.vnet.ibm.com> - 2015-11-23 15:00 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Pratyush Anand <panand@redhat.com> - 2015-11-23 16:20 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Laurent Dufour <ldufour@linux.vnet.ibm.com> - 2015-11-23 16:30 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Ming Lei <ming.lei@canonical.com> - 2015-11-23 16:30 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Laurent Dufour <ldufour@linux.vnet.ibm.com> - 2015-11-23 17:30 +0100
Re: kernel BUG at drivers/scsi/scsi_lib.c:1096! Mark Salter <msalter@redhat.com> - 2015-11-24 02:40 +0100
csiph-web