Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1596274

Re: [PATCH 1/1] x86/cqm: Cqm requirements

Show all headers | View raw

On Thu, Mar 9, 2017 at 3:01 AM, Thomas Gleixner <tglx@linutronix.de> wrote:
> On Wed, 8 Mar 2017, David Carrillo-Cisneros wrote:
>> On Wed, Mar 8, 2017 at 12:30 AM, Thomas Gleixner <tglx@linutronix.de> wrote:
>> > Same applies for per CPU measurements.
>>
>> For CPU measurements. We need perf-like CPU filtering to support tools
>> that perform low overhead monitoring by polling CPU events. These
>> tools approximate per-cgroup/task events by reconciling CPU events
>> with logs of what job run when in what CPU.
>
> Sorry, but for CQM that's just voodoo analysis.

I'll argue that. Yet, perf-like CPU is also needed for MBM, a less
contentious scenario, I believe.


>
> CPU default is CAT group 0      (20% of cache)
> T1 belongs to CAT group 1       (40% of cache)
> T2 belongs to CAT group 2       (40% of cache)
>
> Now you do low overhead samples of the CPU (all groups accounted) with 1
> second period.
>
> Lets assume that T1 runs 50% and T2 runs 20% the rest of the time is
> utilized by random other things and the kernel itself (using CAT group 0).
>
> What is the accumulated value telling you?

In this single example not much, only the sum of occupancies. But
assume I have T1...T10000 different jobs, and I randomly select a pair
of those jobs to run together in a machine, (they become the T1 and T2
in your example). Then I repeat that hundreds of thousands of times.

I can collect all data with (tasks run, time run, occupancy) and build
a simple regression to estimate the expected occupancy (and some
confidence interval). That inaccurate but approximate value is very
useful to feed into a job scheduler. Furthermore, it can be correlated
with values of other events that are currently sampled this way.

>
> How do you approximate that back to T1/T2 and the rest?

Described above for large numbers and random samples. More
sophisticated (voodo?) statistic techniques are employed in practice
to account for almost all issues I could think of (selection bias,
missing values, interaction between tasks, etc). They seem to work
fine.

>
> How do you do that when the tasks are switching between the samples several
> times?

It does not work well for a single run (your example). But for the
example I gave, one can just rely on Random Sampling, Law of Large
Numbers, and Central Limit Theorem.

Thanks,
David

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

RE: [PATCH 1/1] x86/cqm: Cqm requirements "Luck, Tony" <tony.luck@intel.com> - 2017-03-07 21:10 +0100
  RE: [PATCH 1/1] x86/cqm: Cqm requirements Thomas Gleixner <tglx@linutronix.de> - 2017-03-07 21:50 +0100
  Re: [PATCH 1/1] x86/cqm: Cqm requirements Stephane Eranian <eranian@google.com> - 2017-03-08 00:30 +0100
    Re: [PATCH 1/1] x86/cqm: Cqm requirements Shivappa Vikas <vikas.shivappa@intel.com> - 2017-03-08 01:20 +0100
    Re: [PATCH 1/1] x86/cqm: Cqm requirements Thomas Gleixner <tglx@linutronix.de> - 2017-03-08 10:50 +0100
      Re: [PATCH 1/1] x86/cqm: Cqm requirements David Carrillo-Cisneros <davidcc@google.com> - 2017-03-08 19:00 +0100
        Re: [PATCH 1/1] x86/cqm: Cqm requirements Thomas Gleixner <tglx@linutronix.de> - 2017-03-09 12:10 +0100
          Re: [PATCH 1/1] x86/cqm: Cqm requirements David Carrillo-Cisneros <davidcc@google.com> - 2017-03-09 19:10 +0100
            Re: [PATCH 1/1] x86/cqm: Cqm requirements Thomas Gleixner <tglx@linutronix.de> - 2017-03-10 16:00 +0100
              Re: [PATCH 1/1] x86/cqm: Cqm requirements David Carrillo-Cisneros <davidcc@google.com> - 2017-03-11 03:00 +0100
                Re: [PATCH 1/1] x86/cqm: Cqm requirements Thomas Gleixner <tglx@linutronix.de> - 2017-03-13 20:20 +0100
                Re: [PATCH 1/1] x86/cqm: Cqm requirements David Carrillo-Cisneros <davidcc@google.com> - 2017-03-13 21:10 +0100
                Re: [PATCH 1/1] x86/cqm: Cqm requirements Thomas Gleixner <tglx@linutronix.de> - 2017-03-13 21:30 +0100

csiph-web