Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1547758

Re: [PATCH 01/14] x86/cqm: Intel Resource Monitoring Documentation

Show all headers | View raw

On Tue, Dec 27, 2016 at 12:00 PM, Andi Kleen <andi@firstfloor.org> wrote:
> Shivappa Vikas <vikas.shivappa@intel.com> writes:
>>
>> Ok , looks like the interface  is the problem. Will try to fix
>> this. We are just trying to have a light weight monitoring
>> option so that its reasonable to monitor for a
>> very long time (like lifetime of process etc). Mainly to not have all
>> the perf scheduling overhead.
>
> That seems like an odd reason to define a completely new user interface.
> This is to avoid one MSR write for a RMID change per context switch
> in/out cgroup or is it other code too?
>
> Is there some number you can put to the overhead?

I obtained some timing by manually instrumenting the kernel in a Haswell EP.

When using one intel_cmt/llc_occupancy/ cgroup perf_event in one CPU, the
avg time to do __perf_event_task_sched_out + __perf_event_task_sched_in is
~1170ns

most of the time is spend in cgroup ctx switch (~1120ns) .

When using continuous monitoring in CQM driver, the avg time to
find the rmid to write inside of pqr_context switch  is ~16ns

Note that this excludes the MSR write. It's only the overhead of
finding the RMID
to write in PQR_ASSOC. Both paths call the same routine to find the
RMID, so there are
about 1100 ns of overhead in perf_cgroup_switch. By inspection I assume most
of it comes from iterating over the pmu list.

> Or is there some other overhead other than the MSR write
> you're concerned about?

No, that problem is solved with the PQR software cache introduced in the series.


> Perhaps some optimization could be done in the code to make it faster,
> then the new interface wouldn't be needed.

There are some. One in my list is to create a list of pmus with at
least one cgroup event
and use it to iterate over in perf_cgroup_switch, instead of using the
"pmus" list.
The pmus list has grown a lot recently with the addition of all the uncore pmus.

Despite this optimization, it's unlikely that the whole sched_out +
sched_in gets that
close to the 15 ns of the non perf_event approach.

Please note that context switch time for llc_occupancy events has more
impact than for
other events because in order to obtain reliable measurements, the
RMID switch must
be active _all_ the time, not only while the event is read.

>
> FWIW there are some pending changes to context switch that will
> eliminate at least one common MSR write [1]. If that was fixed
> you could do the RMID MSR write "for free"

That may save the need for the PQR software cache in this series, but
won't speed up
the context switch.

Thanks,
David

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH V4 00/14] Cqm2: Intel Cache Monitoring fixes and enhancements Vikas Shivappa <vikas.shivappa@linux.intel.com> - 2016-12-17 00:20 +0100
  [PATCH 03/14] x86/rdt: Add rdt common/cqm compile option Vikas Shivappa <vikas.shivappa@linux.intel.com> - 2016-12-17 00:20 +0100
  [PATCH 05/14] x86/cqm,perf/core: Cgroup support prepare Vikas Shivappa <vikas.shivappa@linux.intel.com> - 2016-12-17 00:20 +0100
  [PATCH 13/14] perf/stat: fix bug in handling events in error state Vikas Shivappa <vikas.shivappa@linux.intel.com> - 2016-12-17 00:20 +0100
  [PATCH 07/14] x86/rdt,cqm: Scheduling support update Vikas Shivappa <vikas.shivappa@linux.intel.com> - 2016-12-17 00:20 +0100
  [PATCH 02/14] x86/cqm: Remove cqm recycling/conflict handling Vikas Shivappa <vikas.shivappa@linux.intel.com> - 2016-12-17 00:20 +0100
  [PATCH 01/14] x86/cqm: Intel Resource Monitoring Documentation Vikas Shivappa <vikas.shivappa@linux.intel.com> - 2016-12-17 00:20 +0100
    Re: [PATCH 01/14] x86/cqm: Intel Resource Monitoring Documentation Peter Zijlstra <peterz@infradead.org> - 2016-12-23 13:40 +0100
      Re: [PATCH 01/14] x86/cqm: Intel Resource Monitoring Documentation Shivappa Vikas <vikas.shivappa@intel.com> - 2016-12-23 20:40 +0100
        Re: [PATCH 01/14] x86/cqm: Intel Resource Monitoring Documentation Peter Zijlstra <peterz@infradead.org> - 2016-12-23 22:00 +0100
          Re: [PATCH 01/14] x86/cqm: Intel Resource Monitoring Documentation Shivappa Vikas <vikas.shivappa@intel.com> - 2016-12-23 22:50 +0100
          Re: [PATCH 01/14] x86/cqm: Intel Resource Monitoring Documentation Shivappa Vikas <vikas.shivappa@intel.com> - 2016-12-25 03:00 +0100
            Re: [PATCH 01/14] x86/cqm: Intel Resource Monitoring Documentation David Carrillo-Cisneros <davidcc@google.com> - 2016-12-27 08:20 +0100
            Re: [PATCH 01/14] x86/cqm: Intel Resource Monitoring Documentation Andi Kleen <andi@firstfloor.org> - 2016-12-27 21:10 +0100
              Re: [PATCH 01/14] x86/cqm: Intel Resource Monitoring Documentation Shivappa Vikas <vikas.shivappa@intel.com> - 2016-12-27 21:30 +0100
                Re: [PATCH 01/14] x86/cqm: Intel Resource Monitoring Documentation David Carrillo-Cisneros <davidcc@google.com> - 2016-12-27 22:40 +0100
              Re: [PATCH 01/14] x86/cqm: Intel Resource Monitoring Documentation David Carrillo-Cisneros <davidcc@google.com> - 2016-12-27 22:40 +0100
                Re: [PATCH 01/14] x86/cqm: Intel Resource Monitoring Documentation Andi Kleen <andi@firstfloor.org> - 2016-12-28 00:20 +0100
                Re: [PATCH 01/14] x86/cqm: Intel Resource Monitoring Documentation David Carrillo-Cisneros <davidcc@google.com> - 2016-12-28 02:30 +0100
                Re: [PATCH 01/14] x86/cqm: Intel Resource Monitoring Documentation Shivappa Vikas <vikas.shivappa@intel.com> - 2016-12-28 21:10 +0100
  [PATCH 06/14] x86/cqm: Add cgroup hierarchical monitoring support Vikas Shivappa <vikas.shivappa@linux.intel.com> - 2016-12-17 00:20 +0100
  [PATCH 11/14] x86/cqm: Add failure on open and read Vikas Shivappa <vikas.shivappa@linux.intel.com> - 2016-12-17 00:20 +0100
    Re: [PATCH 11/14] x86/cqm: Add failure on open and read David Carrillo-Cisneros <davidcc@google.com> - 2016-12-23 13:00 +0100
  [PATCH 08/14] x86/cqm: Add support for monitoring task and cgroup together Vikas Shivappa <vikas.shivappa@linux.intel.com> - 2016-12-17 00:20 +0100

csiph-web