Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1245632 > unrolled thread

Re: [PATCH v6 05/11] block: remove split code in blkdev_issue_{discard,write_same}

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCH v6 05/11] block: remove split code in  blkdev_issue_{discard,write_same} Christoph Hellwig <hch@infradead.org> - 2015-10-13 14:00 +0200
    Re: [PATCH v6 05/11] block: remove split code in blkdev_issue_{discard,write_same} Ming Lin <mlin@kernel.org> - 2015-10-13 19:50 +0200
      Re: [PATCH v6 05/11] block: remove split code in  blkdev_issue_{discard,write_same} Christoph Hellwig <hch@infradead.org> - 2015-10-14 15:30 +0200
        Re: [PATCH v6 05/11] block: remove split code in  blkdev_issue_{discard,write_same}B Keith Busch <keith.busch@intel.com> - 2015-10-14 18:40 +0200
          Re: [PATCH v6 05/11] block: remove split code in  blkdev_issue_{discard,write_same}B Christoph Hellwig <hch@infradead.org> - 2015-10-14 19:00 +0200

#1245632 — Re: [PATCH v6 05/11] block: remove split code in blkdev_issue_{discard,write_same}

On Wed, Aug 12, 2015 at 12:07:15AM -0700, Ming Lin wrote:
> From: Ming Lin <ming.l@ssi.samsung.com>
> 
> The split code in blkdev_issue_{discard,write_same} can go away
> now that any driver that cares does the split. We have to make
> sure bio size doesn't overflow.
> 
> For discard, we set max discard sectors to (1<<31)>>9 to ensure
> it doesn't overflow bi_size and hopefully it is of the proper
> granularity as long as the granularity is a power of two.

This ends up breaking discard on NVMe devices for a me.  An mkfs.xfs
which does a discard of the whole device now hangs the system.
Something in here makes it send discard command that the device doesn't
like and the aborts don't seem to help either, although that might be
an issue with the abort handling in the driver.

Just a heads up for now, once I get a bit more time I'll try to collect
a blktrace to figure out how the commands sent to the driver look
different before and after the patch.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1245978 — Re: [PATCH v6 05/11] block: remove split code in blkdev_issue_{discard,write_same}

On Tue, Oct 13, 2015 at 4:50 AM, Christoph Hellwig <hch@infradead.org> wrote:
> On Wed, Aug 12, 2015 at 12:07:15AM -0700, Ming Lin wrote:
>> From: Ming Lin <ming.l@ssi.samsung.com>
>>
>> The split code in blkdev_issue_{discard,write_same} can go away
>> now that any driver that cares does the split. We have to make
>> sure bio size doesn't overflow.
>>
>> For discard, we set max discard sectors to (1<<31)>>9 to ensure
>> it doesn't overflow bi_size and hopefully it is of the proper
>> granularity as long as the granularity is a power of two.
>
> This ends up breaking discard on NVMe devices for a me.  An mkfs.xfs
> which does a discard of the whole device now hangs the system.
> Something in here makes it send discard command that the device doesn't
> like and the aborts don't seem to help either, although that might be
> an issue with the abort handling in the driver.
>
> Just a heads up for now, once I get a bit more time I'll try to collect
> a blktrace to figure out how the commands sent to the driver look
> different before and after the patch.

I just did a quick test with a Samsung 900G NVMe device.
mkfs.xfs is OK on 4.3-rc5.

What's your device model? I may find a similar one to try.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1246741

On Tue, Oct 13, 2015 at 10:44:11AM -0700, Ming Lin wrote:
> I just did a quick test with a Samsung 900G NVMe device.
> mkfs.xfs is OK on 4.3-rc5.
> 
> What's your device model? I may find a similar one to try.

This is a HGST Ultrastar SN100

Analsys and tentativ fix below:

blktrace for before the commit:

259,0    1        2     0.000002543  2394  G   D 0 + 8388607 [mkfs.xfs]
259,0    1        3     0.000008230  2394  I   D 0 + 8388607 [mkfs.xfs]
259,0    1        4     0.000031090   207  D   D 0 + 8388607 [kworker/1:1H]
259,0    1        5     0.000044869  2394  Q   D 8388607 + 8388607 [mkfs.xfs]
259,0    1        6     0.000045992  2394  G   D 8388607 + 8388607 [mkfs.xfs]
259,0    1        7     0.000049559  2394  I   D 8388607 + 8388607 [mkfs.xfs]
259,0    1        8     0.000061551   207  D   D 8388607 + 8388607 [kworker/1:1H]

.. and so on.

blktrace with the commit:

259,0    2        1     0.000000000  1228  Q   D 0 + 4194304 [mkfs.xfs]
259,0    2        2     0.000002543  1228  G   D 0 + 4194304 [mkfs.xfs]
259,0    2        3     0.000010080  1228  I   D 0 + 4194304 [mkfs.xfs]
259,0    2        4     0.000082187   267  D   D 0 + 4194304 [kworker/2:1H]
259,0    2        5     0.000224869  1228  Q   D 4194304 + 4194304 [mkfs.xfs]
259,0    2        6     0.000225835  1228  G   D 4194304 + 4194304 [mkfs.xfs]
259,0    2        7     0.000229457  1228  I   D 4194304 + 4194304 [mkfs.xfs]
259,0    2        8     0.000238507   267  D   D 4194304 + 4194304 [kworker/2:1H]

So discards are smaller, but better aligned.  Now if I tweak a single
line in blk-lib.c to be able to use all of bi_size I get the old I/O
pattern back and everything works fine again:

diff --git a/block/blk-lib.c b/block/blk-lib.c
index bd40292..65b61dc 100644
--- a/block/blk-lib.c
+++ b/block/blk-lib.c
@@ -82,7 +82,7 @@ int blkdev_issue_discard(struct block_device *bdev, sector_t sector,
 			break;
 		}
 
-		req_sects = min_t(sector_t, nr_sects, MAX_BIO_SECTORS);
+		req_sects = min_t(sector_t, nr_sects, UINT_MAX >> 9);
 		end_sect = sector + req_sects;
 
 		bio->bi_iter.bi_sector = sector;
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1246988 — Re: [PATCH v6 05/11] block: remove split code in blkdev_issue_{discard,write_same}B

On Wed, 14 Oct 2015, Christoph Hellwig wrote:
> Analsys and tentativ fix below:
>
> blktrace for before the commit:
>
> 259,0    1        2     0.000002543  2394  G   D 0 + 8388607 [mkfs.xfs]
> 259,0    1        3     0.000008230  2394  I   D 0 + 8388607 [mkfs.xfs]
> 259,0    1        4     0.000031090   207  D   D 0 + 8388607 [kworker/1:1H]
> 259,0    1        5     0.000044869  2394  Q   D 8388607 + 8388607 [mkfs.xfs]
> 259,0    1        6     0.000045992  2394  G   D 8388607 + 8388607 [mkfs.xfs]
> 259,0    1        7     0.000049559  2394  I   D 8388607 + 8388607 [mkfs.xfs]
> 259,0    1        8     0.000061551   207  D   D 8388607 + 8388607 [kworker/1:1H]
>
> .. and so on.
>
> blktrace with the commit:
>
> 259,0    2        1     0.000000000  1228  Q   D 0 + 4194304 [mkfs.xfs]
> 259,0    2        2     0.000002543  1228  G   D 0 + 4194304 [mkfs.xfs]
> 259,0    2        3     0.000010080  1228  I   D 0 + 4194304 [mkfs.xfs]
> 259,0    2        4     0.000082187   267  D   D 0 + 4194304 [kworker/2:1H]
> 259,0    2        5     0.000224869  1228  Q   D 4194304 + 4194304 [mkfs.xfs]
> 259,0    2        6     0.000225835  1228  G   D 4194304 + 4194304 [mkfs.xfs]
> 259,0    2        7     0.000229457  1228  I   D 4194304 + 4194304 [mkfs.xfs]
> 259,0    2        8     0.000238507   267  D   D 4194304 + 4194304 [kworker/2:1H]
>
> So discards are smaller, but better aligned.  Now if I tweak a single
> line in blk-lib.c to be able to use all of bi_size I get the old I/O
> pattern back and everything works fine again:

I see why the proposal is an improvement, but I don't understand why the
current situation results in a hang. Are we missing some kind of error
recovery in the driver?
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1247003 — Re: [PATCH v6 05/11] block: remove split code in blkdev_issue_{discard,write_same}B

On Wed, Oct 14, 2015 at 04:38:50PM +0000, Keith Busch wrote:
> I see why the proposal is an improvement, but I don't understand why the
> current situation results in a hang. Are we missing some kind of error
> recovery in the driver?

The driver tries to abort the commands and eventually gets into a death
spiral.  I'm still trying to understand what exactly is going on.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web