Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1518280

Re: [PATCH 7/8] blk-wbt: add general throttling mechanism

Show all headers | View raw

On 11/09/2016 01:40 AM, Jan Kara wrote:
>>> So for devices with write cache, you will completely drain the device
>>> before waking anybody waiting to issue new requests. Isn't it too strict?
>>> In particular may_queue() will allow new writers to issue new writes once
>>> we drop below the limit so it can happen that some processes will be
>>> effectively starved waiting in may_queue?
>>
>> It is strict, and perhaps too strict. In testing, it's the only method
>> that's proven to keep the writeback caching devices in check. It will
>> round robin the writers, if we have more, which isn't necessarily a bad
>> thing. Each will get to do a burst of depth writes, then wait for a new
>> one.
>
> Well, I'm more concerned about a situation where one writer does a
> bursty write and blocks sleeping in may_queue(). Another writer
> produces a steady flow of write requests so that never causes the
> write queue to completely drain but that writer also never blocks in
> may_queue() when it starts queueing after write queue has somewhat
> drained because it never submits many requests in parallel. In such
> case the first writer would get starved AFAIU.

I see what you are saying. I can modify the logic to ensure that if we
do have a waiter, we queue up others behind it. That should get rid of
that concern.

> Also I'm not sure why such logic for devices with writeback cache is
> needed. Sure the disk is fast to accept writes but if that causes long
> read latencies, we should scale down the writeback limits so that we
> eventually end up submitting only one write request anyway -
> effectively the same thing as limit=0 - won't we?

Basically we want to avoid getting into that situation. The problem with
write caching is that it takes a while for you to notice that anything
is wrong, and when you do, you are way down in the hole. That causes the
first violations to be pretty bad.

I'm fine with playing with this logic and improving it, but I'd rather
wait for a 2nd series for that.

-- 
Jens Axboe

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCHSET] Throttled buffered writeback Jens Axboe <axboe@fb.com> - 2016-11-01 22:20 +0100
  [PATCH 3/8] writeback: mark background writeback as such Jens Axboe <axboe@fb.com> - 2016-11-01 22:20 +0100
    Re: [PATCH 3/8] writeback: mark background writeback as such Christoph Hellwig <hch@lst.de> - 2016-11-02 16:00 +0100
    Re: [PATCH 3/8] writeback: mark background writeback as such Jan Kara <jack@suse.cz> - 2016-11-05 23:30 +0100
  [PATCH 4/8] writeback: track if we're sleeping on progress in balance_dirty_pages() Jens Axboe <axboe@fb.com> - 2016-11-01 22:20 +0100
    Re: [PATCH 4/8] writeback: track if we're sleeping on progress in         balance_dirty_pages() Christoph Hellwig <hch@lst.de> - 2016-11-02 16:00 +0100
    Re: [PATCH 4/8] writeback: track if we're sleeping on progress in  balance_dirty_pages() Jan Kara <jack@suse.cz> - 2016-11-08 14:10 +0100
  [PATCH 6/8] block: add scalable completion tracking of requests Jens Axboe <axboe@fb.com> - 2016-11-01 22:20 +0100
    Re: [PATCH 6/8] block: add scalable completion tracking of requests Jan Kara <jack@suse.cz> - 2016-11-08 14:40 +0100
      Re: [PATCH 6/8] block: add scalable completion tracking of requests Jan Kara <jack@suse.cz> - 2016-11-09 10:10 +0100
        Re: [PATCH 6/8] block: add scalable completion tracking of requests Jens Axboe <axboe@kernel.dk> - 2016-11-09 21:00 +0100
          Re: [PATCH 6/8] block: add scalable completion tracking of requests Jan Kara <jack@suse.cz> - 2016-11-10 20:40 +0100
  [PATCH 2/8] writeback: add wbc_to_write_flags() Jens Axboe <axboe@fb.com> - 2016-11-01 22:20 +0100
    Re: [PATCH 2/8] writeback: add wbc_to_write_flags() Christoph Hellwig <hch@lst.de> - 2016-11-02 16:00 +0100
  [PATCH 7/8] blk-wbt: add general throttling mechanism Jens Axboe <axboe@fb.com> - 2016-11-01 22:20 +0100
    Re: [PATCH 7/8] blk-wbt: add general throttling mechanism Jan Kara <jack@suse.cz> - 2016-11-08 14:40 +0100
      Re: [PATCH 7/8] blk-wbt: add general throttling mechanism Jens Axboe <axboe@kernel.dk> - 2016-11-08 16:50 +0100
        Re: [PATCH 7/8] blk-wbt: add general throttling mechanism Jan Kara <jack@suse.cz> - 2016-11-09 09:50 +0100
          Re: [PATCH 7/8] blk-wbt: add general throttling mechanism Jens Axboe <axboe@kernel.dk> - 2016-11-09 17:10 +0100
            Re: [PATCH 7/8] blk-wbt: add general throttling mechanism Jens Axboe <axboe@kernel.dk> - 2016-11-09 21:00 +0100
              Re: [PATCH 7/8] blk-wbt: add general throttling mechanism Jan Kara <jack@suse.cz> - 2016-11-10 20:40 +0100
            Re: [PATCH 7/8] blk-wbt: add general throttling mechanism Dave Chinner <david@fromorbit.com> - 2016-11-10 01:10 +0100
  [PATCH 8/8] block: hook up writeback throttling Jens Axboe <axboe@fb.com> - 2016-11-01 22:20 +0100
    Re: [PATCH 8/8] block: hook up writeback throttling Jan Kara <jack@suse.cz> - 2016-11-08 14:50 +0100

csiph-web