Add top down metrics to perf stat

From	Andi Kleen <andi@firstfloor.org>
Newsgroups	linux.kernel
Subject	Add top down metrics to perf stat
Date	2016-05-06 01:10 +0200
Message-ID	<rvAz7-1bh-5@gated-at.bofh.it> (permalink)
Organization	linux.* mail to news gateway

Show all headers | View raw

Note to reviewers: includes both tools and kernel patches.
The kernel patches are at the beginning.

[v2: Address review feedback.
Metrics are now always printed, but colored when crossing threshold.
--topdown implies --metric-only.
Various smaller fixes, see individual patches]
[v3: Add --single-thread option and support it with HT off.
Clean up old HT workaround.
Improve documentation.
Various smaller fixes, see individual patches.]
[v4: Rebased on latest tree]
[v5: Rebased on latest tree. Move debug messages to -vv]
[v6: Rebased. Remove .aggr-per-core and --single-thread to not
break old perf binaries. Put SMT enumeration into
generic topology API.]

This patchkit adds support for TopDown measurements to perf stat
It applies on top of my earlier metrics patchkit, posted
separately.

TopDown is intended to replace the frontend cycles idle/
backend cycles idle metrics in standard perf stat output.
These metrics are not reliable in many workloads,
due to out of order effects.

This implements a new --topdown mode in perf stat
(similar to --transaction) that measures the pipe line
bottlenecks using standardized formulas. The measurement
can be all done with 5 counters (one fixed counter)

The result are four metrics:
FrontendBound, BackendBound, BadSpeculation, Retiring

that describe the CPU pipeline behavior on a high level.

FrontendBound and BackendBound
BadSpeculation is a higher

The full top down methology has many hierarchical metrics.
This implementation only supports level 1 which can be
collected without multiplexing. A full implementation
of top down on top of perf is available in pmu-tools toplev.
(http://github.com/andikleen/pmu-tools)

The current version works on Intel Core CPUs starting
with Sandy Bridge, and Atom CPUs starting with Silvermont.
In principle the generic metrics should be also implementable
on other out of order CPUs.

TopDown level 1 uses a set of abstracted metrics which
are generic to out of order CPU cores (although some
CPUs may not implement all of them):

topdown-total-slots Available slots in the pipeline
topdown-slots-issued Slots issued into the pipeline
topdown-slots-retired Slots successfully retired
topdown-fetch-bubbles Pipeline gaps in the frontend
topdown-recovery-bubbles Pipeline gaps during recovery
from misspeculation

These metrics then allow to compute four useful metrics:
FrontendBound, BackendBound, Retiring, BadSpeculation.

The formulas to compute the metrics are generic, they
only change based on the availability on the abstracted
input values.

The kernel declares the events supported by the current
CPU and perf stat then computes the formulas based on the
available metrics.

Example output:

$ perf stat --topdown -I 1000 cmd
1.000735655 frontend bound retiring bad speculation backend bound
1.000735655 S0-C0 2 47.84% 11.69% 8.37% 32.10%
1.000735655 S0-C1 2 45.53% 11.39% 8.52% 34.56%
2.003978563 S0-C0 2 49.47% 12.22% 8.65% 29.66%
2.003978563 S0-C1 2 47.21% 12.98% 8.77% 31.04%
3.004901432 S0-C0 2 49.35% 12.26% 8.68% 29.70%
3.004901432 S0-C1 2 47.23% 12.67% 8.76% 31.35%
4.005766611 S0-C0 2 48.44% 12.14% 8.59% 30.82%
4.005766611 S0-C1 2 46.07% 12.41% 8.67% 32.85%
5.006580592 S0-C0 2 47.91% 12.08% 8.57% 31.44%
5.006580592 S0-C1 2 45.57% 12.27% 8.63% 33.53%
6.007545125 S0-C0 2 47.45% 12.02% 8.57% 31.96%
6.007545125 S0-C1 2 45.13% 12.17% 8.57% 34.14%
7.008539347 S0-C0 2 47.07% 12.03% 8.61% 32.29%
...

For Level 1 Top Down computes metrics per core instead of per logical CPU
on Core CPUs (On Atom CPUs there is no Hyper Threading and TopDown
is per thread)

In this case perf stat automatically enables --per-core mode and also requires
global mode (-a) and avoiding other filters (no cgroup mode)
One side effect is that this may require root rights or a
kernel.perf_event_paranoid=-1 setting.

Full tree available in
git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-misc perf/top-down-20

Back to linux.kernel | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Add top down metrics to perf stat Andi Kleen <andi@firstfloor.org> - 2016-05-06 01:10 +0200
  [PATCH 05/10] x86, perf: Use new topology_max_smt_threads() in HT leak workaround Andi Kleen <andi@firstfloor.org> - 2016-05-06 01:10 +0200
  [PATCH 10/10] perf, tools, stat: Add extra output of counter values with -vv Andi Kleen <andi@firstfloor.org> - 2016-05-06 01:10 +0200
    Re: [PATCH 10/10] perf, tools, stat: Add extra output of counter  values with -vv Jiri Olsa <jolsa@redhat.com> - 2016-05-12 10:10 +0200
  [PATCH 07/10] perf, tools, stat: Scale values by unit before metrics Andi Kleen <andi@firstfloor.org> - 2016-05-06 01:10 +0200
    Re: [PATCH 07/10] perf, tools, stat: Scale values by unit before  metrics Jiri Olsa <jolsa@redhat.com> - 2016-05-07 21:20 +0200
    [tip:perf/core] perf stat: Scale values by unit before metrics tip-bot for Andi Kleen <tipbot@zytor.com> - 2016-05-10 22:40 +0200
  [PATCH 09/10] perf, tools, stat: Add computation of TopDown formulas Andi Kleen <andi@firstfloor.org> - 2016-05-06 01:10 +0200
  [PATCH 04/10] x86, perf: Add Top Down events to Intel Atom Andi Kleen <andi@firstfloor.org> - 2016-05-06 01:10 +0200
  [PATCH 06/10] perf, tools, stat: Avoid fractional digits for integer scales Andi Kleen <andi@firstfloor.org> - 2016-05-06 01:10 +0200
    Re: [PATCH 06/10] perf, tools, stat: Avoid fractional digits for  integer scales Jiri Olsa <jolsa@redhat.com> - 2016-05-07 21:20 +0200
      Re: [PATCH 06/10] perf, tools, stat: Avoid fractional digits for  integer scales Andi Kleen <andi@firstfloor.org> - 2016-05-07 21:30 +0200
        Re: [PATCH 06/10] perf, tools, stat: Avoid fractional digits for  integer scales Jiri Olsa <jolsa@redhat.com> - 2016-05-11 15:10 +0200
          Re: [PATCH 06/10] perf, tools, stat: Avoid fractional digits for  integer scales Arnaldo Carvalho de Melo <acme@kernel.org> - 2016-05-11 18:50 +0200
  [PATCH 02/10] x86, perf: Support sysfs files depending on SMT status Andi Kleen <andi@firstfloor.org> - 2016-05-06 01:10 +0200
    Re: [PATCH 02/10] x86, perf: Support sysfs files depending on SMT  status Peter Zijlstra <peterz@infradead.org> - 2016-05-09 11:50 +0200
      Re: [PATCH 02/10] x86, perf: Support sysfs files depending on SMT  status Andi Kleen <ak@linux.intel.com> - 2016-05-09 16:30 +0200
        Re: [PATCH 02/10] x86, perf: Support sysfs files depending on SMT  status Peter Zijlstra <peterz@infradead.org> - 2016-05-09 16:40 +0200
    Re: [PATCH 02/10] x86, perf: Support sysfs files depending on SMT  status Ingo Molnar <mingo@kernel.org> - 2016-05-12 10:10 +0200
  [PATCH 03/10] x86, perf: Add Top Down events to Intel Core Andi Kleen <andi@firstfloor.org> - 2016-05-06 01:10 +0200
    Re: [PATCH 03/10] x86, perf: Add Top Down events to Intel Core Peter Zijlstra <peterz@infradead.org> - 2016-05-11 15:30 +0200
    Re: [PATCH 03/10] x86, perf: Add Top Down events to Intel Core Jiri Olsa <jolsa@redhat.com> - 2016-05-11 15:30 +0200
    Re: [PATCH 03/10] x86, perf: Add Top Down events to Intel Core Ingo Molnar <mingo@kernel.org> - 2016-05-12 10:20 +0200
  [PATCH 08/10] perf, tools, stat: Basic support for TopDown in perf stat Andi Kleen <andi@firstfloor.org> - 2016-05-06 01:10 +0200
  Re: Add top down metrics to perf stat Jiri Olsa <jolsa@redhat.com> - 2016-05-12 09:50 +0200

csiph-web