Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1571948 > unrolled thread

RE: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  RE: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes "Luck, Tony" <tony.luck@intel.com> - 2017-02-01 21:10 +0100
    Re: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes David Carrillo-Cisneros <davidcc@google.com> - 2017-02-02 00:20 +0100
      RE: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes "Luck, Tony" <tony.luck@intel.com> - 2017-02-02 18:40 +0100
      RE: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes "Luck, Tony" <tony.luck@intel.com> - 2017-02-02 20:40 +0100
        RE: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes Shivappa Vikas <vikas.shivappa@intel.com> - 2017-02-02 21:20 +0100
        Re: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes David Carrillo-Cisneros <davidcc@google.com> - 2017-02-02 21:30 +0100
          Re: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes "Luck, Tony" <tony.luck@intel.com> - 2017-02-03 00:50 +0100
            Re: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes David Carrillo-Cisneros <davidcc@google.com> - 2017-02-03 02:50 +0100
              Re: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes David Carrillo-Cisneros <davidcc@google.com> - 2017-02-03 03:20 +0100
                Re: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes "Luck, Tony" <tony.luck@intel.com> - 2017-02-03 19:00 +0100
                  Re: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes David Carrillo-Cisneros <davidcc@google.com> - 2017-02-03 22:10 +0100
                    Re: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes "Luck, Tony" <tony.luck@intel.com> - 2017-02-03 23:30 +0100
                  Re: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes Stephane Eranian <eranian@google.com> - 2017-02-07 09:10 +0100
                    Re: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes "Luck, Tony" <tony.luck@intel.com> - 2017-02-07 20:10 +0100
                      Re: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes Stephane Eranian <eranian@google.com> - 2017-02-08 22:40 +0100
                    Re: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes Shivappa Vikas <vikas.shivappa@intel.com> - 2017-02-07 21:30 +0100
      RE: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes "Luck, Tony" <tony.luck@intel.com> - 2017-02-06 20:00 +0100
      RE: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes "Luck, Tony" <tony.luck@intel.com> - 2017-02-06 22:30 +0100
        RE: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes Shivappa Vikas <vikas.shivappa@intel.com> - 2017-02-06 22:40 +0100
          Re: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes David Carrillo-Cisneros <davidcc@google.com> - 2017-02-06 22:50 +0100
        Re: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes David Carrillo-Cisneros <davidcc@google.com> - 2017-02-06 23:20 +0100
          RE: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes "Luck, Tony" <tony.luck@intel.com> - 2017-02-07 00:30 +0100
            Re: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes David Carrillo-Cisneros <davidcc@google.com> - 2017-02-07 01:40 +0100
    Re: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes Andi Kleen <andi@firstfloor.org> - 2017-02-02 01:40 +0100
      Re: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes Andi Kleen <andi@firstfloor.org> - 2017-02-02 02:20 +0100
      Re: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes David Carrillo-Cisneros <davidcc@google.com> - 2017-02-02 02:20 +0100
      RE: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes "Yu, Fenghua" <fenghua.yu@intel.com> - 2017-02-02 02:30 +0100
        RE: [PATCH 00/12] Cqm2: Intel Cache quality monitoring fixes Shivappa Vikas <vikas.shivappa@intel.com> - 2017-02-02 19:00 +0100

Page 2 of 2 — ← Prev page 1 [2]

#1575200

On Mon, Feb 6, 2017 at 1:22 PM, Luck, Tony <tony.luck@intel.com> wrote:
>> 12) Whatever fs or syscall is provided instead of perf syscalls, it
>> should provide total_time_enabled in the way perf does, otherwise is
>> hard to interpret MBM values.
>
> It seems that it is hard to define what we even mean by memory bandwidth.
>
> If you are measuring just one task and you find that the total number of bytes
> read is 1GB at some point, and one second later the total bytes is 2GB, then
> it is clear that the average bandwidth for this process is 1GB/s. If you know
> that the task was only running for 50% of the cycles during that 1s interval,
> you could say that it is doing 2GB/s ... which is I believe what you were
> thinking when you wrote #12 above.

Yes, that's one of the cases.

> But whether that is right depends a
> bit on *why* it only ran 50% of the time. If it was time-sliced out by the
> scheduler ... then it may have been trying to be a 2GB/s app. But if it
> was waiting for packets from the network, then it really is using 1 GB/s.

IMO, "right" means that measured bandwidth and running time are
correct. The *why* is a bigger question.

>
> All bets are off if you are measuring a service that consists of several
> tasks running concurrently. All you can really talk about is the aggregate
> average bandwidth (total bytes / wall-clock time). It makes no sense to
> try and factor in how much cpu time each of the individual tasks got.

cgroup mode gives a per-CPU breakdown of event and running time, the
tool aggregates it into running time vs event count. Both per-cpu
breakdown and the aggregate are useful.

Piggy-backing on perf's cgroup mode would give us all the above for free.

>
> -Tony

[toc] | [prev] | [next] | [standalone]

#1575247

> cgroup mode gives a per-CPU breakdown of event and running time, the
> tool aggregates it into running time vs event count. Both per-cpu
> breakdown and the aggregate are useful.
>
> Piggy-backing on perf's cgroup mode would give us all the above for free.

Do you have some sample output from a perf run on a cgroup measuring a
"normal" event showing what you get?

I think that requires that we still go through perf ->start() and ->stop() functions
to know how much time we spent running.  I thought we were looking at bundling
the RMID updates into the same spot in sched() where we switch the CLOSID.
More or less at the "start" point, but there is no "stop".  If we are switching between
runnable processes, it amounts to pretty much the same thing ... except we bill
to someone all the time instead of having a gap in the context switch where we
stopped billing to the old task and haven't started billing to the new one yet.

But if we idle ... then we don't "stop".  Shouldn't matter much from a measurement
perspective because idle won't use cache or consume bandwidth. But we'd count
that time as "on cpu" for the last process to run.

-Tony

[toc] | [prev] | [next] | [standalone]

#1575281

On Mon, Feb 6, 2017 at 3:27 PM, Luck, Tony <tony.luck@intel.com> wrote:
>> cgroup mode gives a per-CPU breakdown of event and running time, the
>> tool aggregates it into running time vs event count. Both per-cpu
>> breakdown and the aggregate are useful.
>>
>> Piggy-backing on perf's cgroup mode would give us all the above for free.
>
> Do you have some sample output from a perf run on a cgroup measuring a
> "normal" event showing what you get?

# perf stat -I 1000 -e cycles -a -C 0-1 -A -x, -G /
     1.000116648,CPU0,20677864,,cycles,/
     1.000169948,CPU1,24760887,,cycles,/
     2.000453849,CPU0,36120862,,cycles,/
     2.000480259,CPU1,12535575,,cycles,/
     3.000664762,CPU0,7564504,,cycles,/
     3.000692552,CPU1,7307480,,cycles,/

>
> I think that requires that we still go through perf ->start() and ->stop() functions
> to know how much time we spent running.  I thought we were looking at bundling
> the RMID updates into the same spot in sched() where we switch the CLOSID.
> More or less at the "start" point, but there is no "stop".  If we are switching between
> runnable processes, it amounts to pretty much the same thing ... except we bill
> to someone all the time instead of having a gap in the context switch where we
> stopped billing to the old task and haven't started billing to the new one yet.

Another problem is that it will require a perf event all the time for
timing measurements to be consistent with RMID measurements.

The only sane option I can come up is to do timing in RDT the way perf
cgroup does it (keep a per-cpu time that increases with local clock's
delta). A reader can add the times for all CPUs in cpu_mask.

>
> But if we idle ... then we don't "stop".  Shouldn't matter much from a measurement
> perspective because idle won't use cache or consume bandwidth. But we'd count
> that time as "on cpu" for the last process to run.

I may be missing something basic but isn't  __switch_to called when
switching to the idle task? that will update the CLOSID and RMID to
whatever the idle task in on, isnt it?

Thanks,
David

[toc] | [prev] | [next] | [standalone]

#1572095

"Luck, Tony" <tony.luck@intel.com> writes:
> 9)	Measure per logical CPU (pick active RMID in same precedence for task/cpu as CAT picks CLOSID)
> 10)	Put multiple CPUs into a group

I'm not sure this is a real requirement. It's just an optimization,
right? If you can assign policies to threads, you can implicitly set it
per CPU through affinity (or the other way around).
The only benefit would be possibly less context switch overhead,
but if all the thread (including idle) assigned to a CPU have the
same policy it would have the same results.

I suspect dropping this would likely simplify the interface significantly.

-Andi

[toc] | [prev] | [next] | [standalone]

#1572107

> > I'm not sure this is a real requirement. It's just an optimization,
> > right? If you can assign policies to threads, you can implicitly set it
> > per CPU through affinity (or the other way around).
> 
> That's difficult when distinct users/systems do monitoring and system
> management.  What if the cluster manager decides to change affinity
> for a task after the monitoring service has initiated monitoring a CPU
> in the way you describe?

Why would you want to monitor a CPU if you don't know what it is
running?  The results would be meaningless. So you really want
to integrate those two services.

> 
> > The only benefit would be possibly less context switch overhead,
> > but if all the thread (including idle) assigned to a CPU have the
> > same policy it would have the same results.
> 
> I think another of the reasons for the CPU monitoring requirement is
> to monitor interruptions in CPUs running the idle thread. In CAT,

idle threads are just threads, so they could be just exposed
to perf (e.g. combination of pid 0 + cpu filter)


> Also, if perf's like monitoring is supported, it'd allow something like
> 
>   perf stat -e LLC-load,LLC-prefetches,intel_cqm/total_bytes -C 2

This would work without a special API.

-Andi

[toc] | [prev] | [next] | [standalone]

#1572109

On Wed, Feb 1, 2017 at 4:35 PM, Andi Kleen <andi@firstfloor.org> wrote:
> "Luck, Tony" <tony.luck@intel.com> writes:
>> 9)    Measure per logical CPU (pick active RMID in same precedence for task/cpu as CAT picks CLOSID)
>> 10)   Put multiple CPUs into a group
>
> I'm not sure this is a real requirement. It's just an optimization,
> right? If you can assign policies to threads, you can implicitly set it
> per CPU through affinity (or the other way around).

That's difficult when distinct users/systems do monitoring and system
management.  What if the cluster manager decides to change affinity
for a task after the monitoring service has initiated monitoring a CPU
in the way you describe?

> The only benefit would be possibly less context switch overhead,
> but if all the thread (including idle) assigned to a CPU have the
> same policy it would have the same results.

I think another of the reasons for the CPU monitoring requirement is
to monitor interruptions in CPUs running the idle thread. In CAT,
those interruptions use the CPU's CLOSID. Here they'd use the CPU's
RMID. Since RMID's are scarce, CPUs can be aggregated into groups to
save many.

Also, if perf's like monitoring is supported, it'd allow something like

  perf stat -e LLC-load,LLC-prefetches,intel_cqm/total_bytes -C 2

Thanks,
David

[toc] | [prev] | [next] | [standalone]

#1572110

> From: Andi Kleen [mailto:andi@firstfloor.org]
> "Luck, Tony" <tony.luck@intel.com> writes:
> > 9)	Measure per logical CPU (pick active RMID in same precedence for
> task/cpu as CAT picks CLOSID)
> > 10)	Put multiple CPUs into a group
> 
> I'm not sure this is a real requirement. It's just an optimization, right? If you
> can assign policies to threads, you can implicitly set it per CPU through affinity
> (or the other way around).
> The only benefit would be possibly less context switch overhead, but if all
> the thread (including idle) assigned to a CPU have the same policy it would
> have the same results.
> 
> I suspect dropping this would likely simplify the interface significantly.

Assigning a pid P to a CPU and monitoring the P don't count all events happening on the CPU.
Other processes/threads (e.g. kernel threads) than the assigned P can run on the CPU.
Monitoring P assigned to the CPU is not equal to monitoring the CPU in a lot cases.

Thanks.

-Fenghua

[toc] | [prev] | [next] | [standalone]

#1572556

On Wed, 1 Feb 2017, Yu, Fenghua wrote:

>> From: Andi Kleen [mailto:andi@firstfloor.org]
>> "Luck, Tony" <tony.luck@intel.com> writes:
>>> 9)	Measure per logical CPU (pick active RMID in same precedence for
>> task/cpu as CAT picks CLOSID)
>>> 10)	Put multiple CPUs into a group
>>
>> I'm not sure this is a real requirement. It's just an optimization, right? If you
>> can assign policies to threads, you can implicitly set it per CPU through affinity
>> (or the other way around).
>> The only benefit would be possibly less context switch overhead, but if all
>> the thread (including idle) assigned to a CPU have the same policy it would
>> have the same results.
>>
>> I suspect dropping this would likely simplify the interface significantly.
>
> Assigning a pid P to a CPU and monitoring the P don't count all events happening on the CPU.
> Other processes/threads (e.g. kernel threads) than the assigned P can run on the CPU.
> Monitoring P assigned to the CPU is not equal to monitoring the CPU in a lot cases.

This matches the use case where a bunch of real time tasks which have no CLOS 
id(kernel threads or others in root group) would want to run exclusively on a 
cpu and are configured so. If any other tasks run there from other class of 
service we dont want to pullute the cache - hence choose their own CLOSId.

Now in order to measure this RMIds need to match the same policy as CAT.

Thanks,
Vikas

>
> Thanks.
>
> -Fenghua
>

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | linux.kernel

csiph-web