Re: [PATCH V3 00/11] block-throttle: add .high limit

From	Vivek Goyal <vgoyal@redhat.com>
Newsgroups	linux.kernel
Subject	Re: [PATCH V3 00/11] block-throttle: add .high limit
Date	2016-10-04 15:30 +0200
Message-ID	<soxXb-3fH-7@gated-at.bofh.it> (permalink)
References	<soiY9-1K3-3@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

On Mon, Oct 03, 2016 at 02:20:19PM -0700, Shaohua Li wrote:
> Hi,
> 
> The background is we don't have an ioscheduler for blk-mq yet, so we can't
> prioritize processes/cgroups.

So this is an interim solution till we have ioscheduler for blk-mq?

> This patch set tries to add basic arbitration
> between cgroups with blk-throttle. It adds a new limit io.high for
> blk-throttle. It's only for cgroup2.
> 
> io.max is a hard limit throttling. cgroups with a max limit never dispatch more
> IO than their max limit. While io.high is a best effort throttling. cgroups
> with high limit can run above their high limit at appropriate time.
> Specifically, if all cgroups reach their high limit, all cgroups can run above
> their high limit. If any cgroup runs under its high limit, all other cgroups
> will run according to their high limit.

Hi Shaohua,

I still don't understand why we should not implement a weight based
proportional IO mechanism and how this mechanism is better than proportional IO .

Agreed that we have issues with proportional IO and we don't have good
solutions for these problems. But I can't see that how this mechanism
will overcome these problems either.

IIRC, biggest issue with proportional IO was that a low prio group might
fill up the device queue with plenty of IO requests and later when high
prio cgroup comes, it will still experience latencies anyway. And solution
to the problem probably would be to get some awareness in device about 
priority of request and map weights to those priority. That way higher
prio requests get prioritized.

Or run device at lower queue depth. That will improve latencies but migth
reduce overall throughput.

Or thorottle number of buffered writes (as Jens's writeback throttling)
patches were doing. Buffered writes seem to be biggest culprit for 
increased latencies and being able to control these should help.

ioprio/weight based proportional IO mechanism is much more generic and
much easier to configure for any kind of storage. io.high is absolute
limit and makes it much harder to configure. One needs to know a lot
about underlying volume/device's bandwidth (which varies a lot anyway
based on workload).

IMHO, we seem to be trying to cater to one specific use case using
this mechanism. Something ioprio/weight based will be much more
generic and we should explore implementing that along with building
notion of ioprio in devices. When these two work together, we might
be able to see good results. Just software mechanism alone might not
be enough.

Vivek

Thread

[PATCH V3 00/11] block-throttle: add .high limit Shaohua Li <shli@fb.com> - 2016-10-03 23:30 +0200
  [PATCH v3 10/11] block-throttle: add a simple idle detection Shaohua Li <shli@fb.com> - 2016-10-03 23:30 +0200
  [PATCH v3 07/11] blk-throttle: make throtl_slice tunable Shaohua Li <shli@fb.com> - 2016-10-03 23:30 +0200
  [PATCH v3 08/11] blk-throttle: detect completed idle cgroup Shaohua Li <shli@fb.com> - 2016-10-03 23:30 +0200
  [PATCH v3 06/11] blk-throttle: make sure expire time isn't too big Shaohua Li <shli@fb.com> - 2016-10-03 23:30 +0200
  [PATCH v3 11/11] blk-throttle: ignore idle cgroup limit Shaohua Li <shli@fb.com> - 2016-10-03 23:30 +0200
  [PATCH v3 09/11] block-throttle: make bandwidth change smooth Shaohua Li <shli@fb.com> - 2016-10-03 23:30 +0200
  [PATCH v3 03/11] block-throttle: configure bps/iops limit for cgroup in high limit Shaohua Li <shli@fb.com> - 2016-10-03 23:30 +0200
  [PATCH v3 01/11] block-throttle: prepare support multiple limits Shaohua Li <shli@fb.com> - 2016-10-03 23:30 +0200
  [PATCH v3 02/11] block-throttle: add .high interface Shaohua Li <shli@fb.com> - 2016-10-03 23:30 +0200
  Re: [PATCH V3 00/11] block-throttle: add .high limit Vivek Goyal <vgoyal@redhat.com> - 2016-10-04 15:30 +0200
    Re: [PATCH V3 00/11] block-throttle: add .high limit Tejun Heo <tj@kernel.org> - 2016-10-04 18:00 +0200
      Re: [PATCH V3 00/11] block-throttle: add .high limit Tejun Heo <tj@kernel.org> - 2016-10-04 18:30 +0200
        Re: [PATCH V3 00/11] block-throttle: add .high limit Paolo Valente <paolo.valente@unimore.it> - 2016-10-04 19:10 +0200
          Re: [PATCH V3 00/11] block-throttle: add .high limit Paolo Valente <paolo.valente@unimore.it> - 2016-10-04 19:50 +0200
            Re: [PATCH V3 00/11] block-throttle: add .high limit Tejun Heo <tj@kernel.org> - 2016-10-04 21:00 +0200
              Re: [PATCH V3 00/11] block-throttle: add .high limit Paolo Valente <paolo.valente@unimore.it> - 2016-10-04 21:10 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit Tejun Heo <tj@kernel.org> - 2016-10-04 21:20 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit Paolo Valente <paolo.valente@unimore.it> - 2016-10-04 21:40 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit Tejun Heo <tj@kernel.org> - 2016-10-04 22:30 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit Paolo Valente <paolo.valente@unimore.it> - 2016-10-05 14:40 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit Vivek Goyal <vgoyal@redhat.com> - 2016-10-05 15:20 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit Paolo Valente <paolo.valente@unimore.it> - 2016-10-05 16:10 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit Tejun Heo <tj@kernel.org> - 2016-10-05 17:00 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit Paolo Valente <paolo.valente@unimore.it> - 2016-10-05 21:50 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit Paolo Valente <paolo.valente@unimore.it> - 2016-10-05 22:10 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit Paolo Valente <paolo.valente@unimore.it> - 2016-10-06 10:00 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit Paolo Valente <paolo.valente@unimore.it> - 2016-10-06 15:20 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit Vivek Goyal <vgoyal@redhat.com> - 2016-10-06 19:50 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit Paolo Valente <paolo.valente@unimore.it> - 2016-10-06 20:10 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit Vivek Goyal <vgoyal@redhat.com> - 2016-10-06 20:40 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit Paolo Valente <paolo.valente@unimore.it> - 2016-10-06 23:00 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit Mark Brown <broonie@kernel.org> - 2016-10-06 21:50 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit Paolo Valente <paolo.valente@unimore.it> - 2016-10-07 00:30 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit Paolo Valente <paolo.valente@unimore.it> - 2016-10-05 22:00 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit Paolo Valente <paolo.valente@unimore.it> - 2016-10-06 09:30 +0200
                Fwd: [PATCH V3 00/11] block-throttle: add .high limit Kyle Sanderson <kyle.leet@gmail.com> - 2016-10-09 03:20 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit Linus Walleij <linus.walleij@linaro.org> - 2016-10-06 10:10 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit Mark Brown <broonie@kernel.org> - 2016-10-06 13:10 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit "Austin S. Hemmelgarn" <ahferroin7@gmail.com> - 2016-10-06 14:00 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit Paolo Valente <paolo.valente@unimore.it> - 2016-10-06 15:00 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit "Austin S. Hemmelgarn" <ahferroin7@gmail.com> - 2016-10-06 16:00 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit Paolo Valente <paolo.valente@unimore.it> - 2016-10-06 17:10 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit "Austin S. Hemmelgarn" <ahferroin7@gmail.com> - 2016-10-06 17:20 +0200
                Re: [PATCH V3 00/11] block-throttle: add .high limit Heinz Diehl <htd+ml@fritha.org> - 2016-10-08 12:50 +0200
            Re: [PATCH V3 00/11] block-throttle: add .high limit Paolo Valente <paolo.valente@unimore.it> - 2016-10-04 21:50 +0200
      Re: [PATCH V3 00/11] block-throttle: add .high limit Paolo Valente <paolo.valente@unimore.it> - 2016-10-04 18:30 +0200
      Re: [PATCH V3 00/11] block-throttle: add .high limit Vivek Goyal <vgoyal@redhat.com> - 2016-10-04 20:20 +0200
        Re: [PATCH V3 00/11] block-throttle: add .high limit Tejun Heo <tj@kernel.org> - 2016-10-04 21:00 +0200
          Re: [PATCH V3 00/11] block-throttle: add .high limit Paolo Valente <paolo.valente@unimore.it> - 2016-10-04 21:00 +0200

csiph-web