Re: [PATCH 5/6] f2fs: add a kernel thread to issue discard commands asynchronously

From	Christoph Hellwig <hch@infradead.org>
Newsgroups	linux.kernel
Subject	Re: [PATCH 5/6] f2fs: add a kernel thread to issue discard commands asynchronously
Date	2017-02-08 17:50 +0100
Message-ID	<t8DBo-2Hv-25@gated-at.bofh.it> (permalink)
References	(2 earlier) <sZ55U-6Ta-23@gated-at.bofh.it> <sZfyi-4If-9@gated-at.bofh.it> <t0jq9-3zg-1@gated-at.bofh.it> <t7qZz-5l9-13@gated-at.bofh.it> <t84X0-5LX-7@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

On Mon, Feb 06, 2017 at 07:44:03PM -0800, Jaegeuk Kim wrote:
> Sorry for the late response due to the travel.
> 
> When doing fstrim with a fresh f2fs image fomatted on Intel NVMe SSD whose
> model name is SSDPE2MW012T4, I've got the following trace.

<snip>

> So, I investigated why block_rq_complete() happened in more detail.
> 
> The root-caused call path looks like:
>  - submit_bio
>   - generic_make_request
>    - q->make_request_fn
>     - blk_mq_make_request
>      - blk_mq_map_request
>       - blk_mq_alloc_request
>        - blk_mq_get_tag
>         - __blk_mq_get_tag
>          - bt_get
>           - blk_mq_run_hw_queue
>           - finish_wait
>           --> this waits for pending 8 discard bios!

You're blocking on tag allocation.  How many tags per queue does
your device have?, e.g. do a

cat /sys/block/nvme0n1/mq/0/nr_tags

> It seems the problem comes from the storage processing discard commands too
> slowly comparing to normal read/write IOs.
> 
> Any thoughts?

Deallocate is always going to be an exception path compared to normal
read/write… but just how much slower is going to be device
dependent.

One option would be to reuse the number of discards, for that can you
try the series here to support vectored discards:

http://git.infradead.org/users/hch/block.git/shortlog/refs/heads/vectored-discard-for-axboe

Thread

Re: [PATCH 5/6] f2fs: add a kernel thread to issue discard commands  asynchronously Christoph Hellwig <hch@infradead.org> - 2017-02-05 10:10 +0100
  Re: [PATCH 5/6] f2fs: add a kernel thread to issue discard commands  asynchronously Jaegeuk Kim <jaegeuk@kernel.org> - 2017-02-07 04:50 +0100
    Re: [PATCH 5/6] f2fs: add a kernel thread to issue discard commands  asynchronously Christoph Hellwig <hch@infradead.org> - 2017-02-08 17:50 +0100
      Re: [PATCH 5/6] f2fs: add a kernel thread to issue discard commands  asynchronously Jaegeuk Kim <jaegeuk@kernel.org> - 2017-02-08 23:10 +0100

csiph-web