Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1620026 > unrolled thread

Re: bfq-mq performance comparison to cfq

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: bfq-mq performance comparison to cfq Bart Van Assche <Bart.VanAssche@sandisk.com> - 2017-04-10 17:20 +0200
    Re: bfq-mq performance comparison to cfq Paolo Valente <paolo.valente@linaro.org> - 2017-04-11 09:40 +0200
      Re: bfq-mq performance comparison to cfq Bart Van Assche <Bart.VanAssche@sandisk.com> - 2017-04-19 07:10 +0200
        Re: bfq-mq performance comparison to cfq Paolo Valente <paolo.valente@linaro.org> - 2017-04-19 09:10 +0200
          Re: bfq-mq performance comparison to cfq Bart Van Assche <Bart.VanAssche@sandisk.com> - 2017-04-19 17:50 +0200

#1620026 — Re: bfq-mq performance comparison to cfq

On Mon, 2017-04-10 at 11:55 +0200, Paolo Valente wrote:
> That said, if you do always want maximum throughput, even at the
> expense of latency, then just switch off low-latency heuristics, i.e.,
> set low_latency to 0.  Depending on the device, setting slice_ilde to
> 0 may help a lot too (as well as with CFQ).  If the throughput is
> still low also after forcing BFQ to an only-throughput mode, then you
> hit some bug, and I'll have a little more work to do ...

Hello Paolo,

Has it been considered to make applications tell the I/O scheduler
whether to optimize for latency or for throughput? It shouldn't be that
hard for window managers and shells to figure out whether or not a new
application that is being started is interactive or not. This would
require a mechanism that allows applications to provide such information
to the I/O scheduler. Wouldn't that be a better approach than the I/O
scheduler trying to guess whether or not an application is an interactive
application?

Bart.

[toc] | [next] | [standalone]

#1620983

> Il giorno 10 apr 2017, alle ore 17:15, Bart Van Assche <bart.vanassche@sandisk.com> ha scritto:
> 
> On Mon, 2017-04-10 at 11:55 +0200, Paolo Valente wrote:
>> That said, if you do always want maximum throughput, even at the
>> expense of latency, then just switch off low-latency heuristics, i.e.,
>> set low_latency to 0.  Depending on the device, setting slice_ilde to
>> 0 may help a lot too (as well as with CFQ).  If the throughput is
>> still low also after forcing BFQ to an only-throughput mode, then you
>> hit some bug, and I'll have a little more work to do ...
> 
> Hello Paolo,
> 
> Has it been considered to make applications tell the I/O scheduler
> whether to optimize for latency or for throughput? It shouldn't be that
> hard for window managers and shells to figure out whether or not a new
> application that is being started is interactive or not. This would
> require a mechanism that allows applications to provide such information
> to the I/O scheduler. Wouldn't that be a better approach than the I/O
> scheduler trying to guess whether or not an application is an interactive
> application?
> 

IMO that would be an (or maybe the) optimal solution, in terms of both
throughput and latency.  We have even developed a prototype doing what
you propose, for Android.  Unfortunately, I have not yet succeeded in
getting support, to turn it into candidate production code, or to make
a similar solution for lsb-compliant systems.

Thanks,
Paolo


> Bart.

[toc] | [prev] | [next] | [standalone]

#1625817

On 04/11/17 00:29, Paolo Valente wrote:
>
>> Il giorno 10 apr 2017, alle ore 17:15, Bart Van Assche <bart.vanassche@sandisk.com> ha scritto:
>>
>> On Mon, 2017-04-10 at 11:55 +0200, Paolo Valente wrote:
>>> That said, if you do always want maximum throughput, even at the
>>> expense of latency, then just switch off low-latency heuristics, i.e.,
>>> set low_latency to 0.  Depending on the device, setting slice_ilde to
>>> 0 may help a lot too (as well as with CFQ).  If the throughput is
>>> still low also after forcing BFQ to an only-throughput mode, then you
>>> hit some bug, and I'll have a little more work to do ...
>>
>> Has it been considered to make applications tell the I/O scheduler
>> whether to optimize for latency or for throughput? It shouldn't be that
>> hard for window managers and shells to figure out whether or not a new
>> application that is being started is interactive or not. This would
>> require a mechanism that allows applications to provide such information
>> to the I/O scheduler. Wouldn't that be a better approach than the I/O
>> scheduler trying to guess whether or not an application is an interactive
>> application?
>
> IMO that would be an (or maybe the) optimal solution, in terms of both
> throughput and latency.  We have even developed a prototype doing what
> you propose, for Android.  Unfortunately, I have not yet succeeded in
> getting support, to turn it into candidate production code, or to make
> a similar solution for lsb-compliant systems.

Hello Paolo,

What API was used by the Android application to tell the I/O scheduler 
to optimize for latency? Do you think that it would be sufficient if the 
application uses the ioprio_set() system call to set the I/O priority to 
IOPRIO_CLASS_RT?

Thanks,

Bart.

[toc] | [prev] | [next] | [standalone]

#1625869

> Il giorno 19 apr 2017, alle ore 07:01, Bart Van Assche <bart.vanassche@sandisk.com> ha scritto:
> 
> On 04/11/17 00:29, Paolo Valente wrote:
>> 
>>> Il giorno 10 apr 2017, alle ore 17:15, Bart Van Assche <bart.vanassche@sandisk.com> ha scritto:
>>> 
>>> On Mon, 2017-04-10 at 11:55 +0200, Paolo Valente wrote:
>>>> That said, if you do always want maximum throughput, even at the
>>>> expense of latency, then just switch off low-latency heuristics, i.e.,
>>>> set low_latency to 0.  Depending on the device, setting slice_ilde to
>>>> 0 may help a lot too (as well as with CFQ).  If the throughput is
>>>> still low also after forcing BFQ to an only-throughput mode, then you
>>>> hit some bug, and I'll have a little more work to do ...
>>> 
>>> Has it been considered to make applications tell the I/O scheduler
>>> whether to optimize for latency or for throughput? It shouldn't be that
>>> hard for window managers and shells to figure out whether or not a new
>>> application that is being started is interactive or not. This would
>>> require a mechanism that allows applications to provide such information
>>> to the I/O scheduler. Wouldn't that be a better approach than the I/O
>>> scheduler trying to guess whether or not an application is an interactive
>>> application?
>> 
>> IMO that would be an (or maybe the) optimal solution, in terms of both
>> throughput and latency.  We have even developed a prototype doing what
>> you propose, for Android.  Unfortunately, I have not yet succeeded in
>> getting support, to turn it into candidate production code, or to make
>> a similar solution for lsb-compliant systems.
> 
> Hello Paolo,
> 
> What API was used by the Android application to tell the I/O scheduler 
> to optimize for latency? Do you think that it would be sufficient if the 
> application uses the ioprio_set() system call to set the I/O priority to 
> IOPRIO_CLASS_RT?
> 

That's exactly the hack we are using in our prototype.  However, it
can only be a temporary hack, because it mixes two slightly different
concepts: 1) the activation of weight raising and other mechanisms for
reducing latency for the target app, 2) the assignment of a different
priority class, which (cleanly) means just that processes in a lower
priority class will be served only when the processes of the target
app have no pending I/O request.  Finding a clean boosting API would
be one of the main steps to turn our prototype into a usable solution.

Thanks,
Paolo

> Thanks,
> 
> Bart.

[toc] | [prev] | [next] | [standalone]

#1626409

On Wed, 2017-04-19 at 09:02 +0200, Paolo Valente wrote:
> > Il giorno 19 apr 2017, alle ore 07:01, Bart Van Assche <bart.vanassche@sandisk.com> ha scritto:
> > What API was used by the Android application to tell the I/O scheduler 
> > to optimize for latency? Do you think that it would be sufficient if the 
> > application uses the ioprio_set() system call to set the I/O priority to 
> > IOPRIO_CLASS_RT?
> 
> That's exactly the hack we are using in our prototype.  However, it
> can only be a temporary hack, because it mixes two slightly different
> concepts: 1) the activation of weight raising and other mechanisms for
> reducing latency for the target app, 2) the assignment of a different
> priority class, which (cleanly) means just that processes in a lower
> priority class will be served only when the processes of the target
> app have no pending I/O request.  Finding a clean boosting API would
> be one of the main steps to turn our prototype into a usable solution.

Hello Paolo,

Sorry but I do not agree that you call this use of I/O priorities a hack.
I also do not agree that I/O requests submitted by processes in a lower
priority class will only be served by the I/O scheduler when there are no
pending requests in a higher class. It wouldn't be that hard to modify I/O
schedulers that support I/O priorities to avoid the starvation you referred
to. What I expect that will happen is that sooner or later a Linux
distributor will start receiving bug reports about the heuristics for
detecting interactive and streaming applications and that the person who
will work on that bug report will realize that it will be easier to remove
those heuristics from BFQ and to modify streaming applications and the
software that starts interactive applications (e.g. a window manager) to
use a higher I/O priority.

Please also note that what I described above may require to introduce
additional I/O priorities in the Linux kernel next to the existing I/O
priorities RT, BE and NONE and that this may require to map multiple of
these priorities onto the same drive priority.

Bart.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web