Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1724527 > unrolled thread
| Started by | Andi Kleen <andi@firstfloor.org> |
|---|---|
| First post | 2017-08-31 21:50 +0200 |
| Last post | 2017-09-01 20:00 +0200 |
| Articles | 20 on this page of 28 — 4 participants |
Back to article view | Back to linux.kernel
Support standalone metrics and metric groups for perf Andi Kleen <andi@firstfloor.org> - 2017-08-31 21:50 +0200
[PATCH v3 07/11] perf, tools, list: Add metric groups to perf list Andi Kleen <andi@firstfloor.org> - 2017-08-31 21:50 +0200
[PATCH v3 11/11] perf, tools, stat: Update walltime_nsecs_stats in interval mode Andi Kleen <andi@firstfloor.org> - 2017-08-31 21:50 +0200
[PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat Andi Kleen <andi@firstfloor.org> - 2017-08-31 21:50 +0200
Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat Arnaldo Carvalho de Melo <acme@kernel.org> - 2017-09-04 19:20 +0200
Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat Andi Kleen <andi@firstfloor.org> - 2017-09-04 19:40 +0200
Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat Arnaldo Carvalho de Melo <acme@kernel.org> - 2017-09-05 20:10 +0200
Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat Arnaldo Carvalho de Melo <acme@kernel.org> - 2017-09-05 20:20 +0200
Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat Arnaldo Carvalho de Melo <acme@kernel.org> - 2017-09-05 20:40 +0200
Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat Andi Kleen <ak@linux.intel.com> - 2017-09-05 20:30 +0200
Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat Arnaldo Carvalho de Melo <acme@kernel.org> - 2017-09-05 21:00 +0200
Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat Andi Kleen <andi@firstfloor.org> - 2017-09-05 22:00 +0200
Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat Arnaldo Carvalho de Melo <acme@kernel.org> - 2017-09-05 22:10 +0200
Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat Andi Kleen <ak@linux.intel.com> - 2017-09-05 22:40 +0200
Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat Arnaldo Carvalho de Melo <acme@kernel.org> - 2017-09-08 20:20 +0200
Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat Andi Kleen <ak@linux.intel.com> - 2017-09-08 21:10 +0200
[PATCH v3 04/11] perf, tools: Print generic metric header even for failed expressions Andi Kleen <andi@firstfloor.org> - 2017-08-31 21:50 +0200
[PATCH v3 10/11] perf, tools, stat: Hide internal duration_time counter Andi Kleen <andi@firstfloor.org> - 2017-08-31 21:50 +0200
[PATCH v3 09/11] perf, tools, stat: Support duration_time for metrics Andi Kleen <andi@firstfloor.org> - 2017-08-31 21:50 +0200
[PATCH v3 01/11] perf, tools: Support weak groups Andi Kleen <andi@firstfloor.org> - 2017-08-31 21:50 +0200
Re: [PATCH v3 01/11] perf, tools: Support weak groups Jiri Olsa <jolsa@redhat.com> - 2017-09-01 19:00 +0200
Re: [PATCH v3 01/11] perf, tools: Support weak groups Jiri Olsa <jolsa@redhat.com> - 2017-09-01 19:10 +0200
Re: [PATCH v3 01/11] perf, tools: Support weak groups Arnaldo Carvalho de Melo <acme@kernel.org> - 2017-09-04 19:00 +0200
[PATCH v3 08/11] perf, tools, stat: Don't use ctx for saved values lookup Andi Kleen <andi@firstfloor.org> - 2017-08-31 21:50 +0200
Re: Support standalone metrics and metric groups for perf Jiri Olsa <jolsa@redhat.com> - 2017-09-01 19:30 +0200
Re: Support standalone metrics and metric groups for perf Jiri Olsa <jolsa@redhat.com> - 2017-09-01 19:40 +0200
Re: Support standalone metrics and metric groups for perf Andi Kleen <andi@firstfloor.org> - 2017-09-01 19:50 +0200
Re: Support standalone metrics and metric groups for perf Jiri Olsa <jolsa@redhat.com> - 2017-09-01 20:00 +0200
Page 1 of 2 [1] 2 Next page →
| From | Andi Kleen <andi@firstfloor.org> |
|---|---|
| Date | 2017-08-31 21:50 +0200 |
| Subject | Support standalone metrics and metric groups for perf |
| Message-ID | <ukDDr-1ub-3@gated-at.bofh.it> |
Add generic support for standalone metrics specified in JSON files
to perf stat. A metric is a formula that uses multiple events
to compute a higher level result (e.g. IPC).
For more complex metrics we need to have micro architecture
specific knowledge, so it makes sense to tie metrics to
JSON event lists.
Previously metrics were always tied to an event and automatically
enabled with that event. But now change it that we can have
standalone metrics. They are in the same JSON data structure
as events, but don't have an event name, only a metric name.
We also allow to organize the metrics in metric groups, which
allows a short cut to select several related metrics at once.
This patch kit adds the code to perf to manage metric groups
The first few patches are generic bug fixes and can be applied
directly. Then there is a 'weak group' feature that is useful
independently from metrics. After there are metrics specific
patches.
The patches are available in
git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-misc.git perf/metric-group-6
The actual Intel JSON metrics are available in git as a separate pull
request in
git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-misc.git perf/intel-json-metrics-2
Some example output:
% perf list metricgroup
..
Metric Groups:
DSB:
DSB_Coverage
[Fraction of Uops delivered by the DSB (aka Decoded Icache; or Uop Cache)]
FLOPS:
GFLOPs
[Giga Floating Point Operations Per Second]
Frontend:
IFetch_Line_Utilization
[Rough Estimation of fraction of fetched lines bytes that were likely consumed by program instructions]
Frontend_Bandwidth:
DSB_Coverage
[Fraction of Uops delivered by the DSB (aka Decoded Icache; or Uop Cache)]
Memory_BW:
MLP
[Memory-Level-Parallelism (average number of L1 miss demand load when there is at least 1 such miss)]
% perf stat -M Summary --metric-only -a sleep 1
Performance counter stats for 'system wide':
Instructions CLKS CPU_Utilization GFLOPs SMT_2T_Utilization Kernel_Utilization
317614222.0 1392930775.0 0.0 0.0 0.2 0.1
1.001497549 seconds time elapsed
% perf stat -M GFLOPs flops
Performance counter stats for 'flops':
3,999,541,471 fp_comp_ops_exe.sse_scalar_single # 1.2 GFLOPs (66.65%)
14 fp_comp_ops_exe.sse_scalar_double (66.65%)
0 fp_comp_ops_exe.sse_packed_double (66.67%)
0 fp_comp_ops_exe.sse_packed_single (66.70%)
0 simd_fp_256.packed_double (66.70%)
0 simd_fp_256.packed_single (66.67%)
3.238372845 seconds time elapsed
v1: Initial post
v2: Address all review feedback (see individual patches)
BPF now works again.
Fix some bugs in perf list printing that I added last minute last time.
v3: Address all review feedback. Some patches are split. Rebased.
Not caching cpuids because it's too complicated.
[toc] | [next] | [standalone]
| From | Andi Kleen <andi@firstfloor.org> |
|---|---|
| Date | 2017-08-31 21:50 +0200 |
| Subject | [PATCH v3 07/11] perf, tools, list: Add metric groups to perf list |
| Message-ID | <ukDDs-1ub-25@gated-at.bofh.it> |
| In reply to | #1724527 |
From: Andi Kleen <ak@linux.intel.com>
Add code to perf list to print metric groups, and metrics
that don't have an event name. The metricgroup code collects
the eventgroups and events into a rblist, and then prints
them according to the configured filters.
The metricgroups are printed by default, but can be
limited by perf list metric or perf list metricgroup
% perf list metricgroup
..
Metric Groups:
DSB:
DSB_Coverage
[Fraction of Uops delivered by the DSB (aka Decoded Icache; or Uop Cache)]
FLOPS:
GFLOPs
[Giga Floating Point Operations Per Second]
Frontend:
IFetch_Line_Utilization
[Rough Estimation of fraction of fetched lines bytes that were likely consumed by program instructions]
Frontend_Bandwidth:
DSB_Coverage
[Fraction of Uops delivered by the DSB (aka Decoded Icache; or Uop Cache)]
Memory_BW:
MLP
[Memory-Level-Parallelism (average number of L1 miss demand load when there is at least 1 such miss)]
v2: Check return value of asprintf to fix warning on FC26
Fix key in lookup/addition for the groups list
Signed-off-by: Andi Kleen <ak@linux.intel.com>
---
tools/perf/Documentation/perf-list.txt | 7 +-
tools/perf/builtin-list.c | 7 ++
tools/perf/util/metricgroup.c | 176 +++++++++++++++++++++++++++++++++
tools/perf/util/parse-events.c | 3 +
4 files changed, 192 insertions(+), 1 deletion(-)
diff --git a/tools/perf/Documentation/perf-list.txt b/tools/perf/Documentation/perf-list.txt
index d432965d728d..ca6369fd83c1 100644
--- a/tools/perf/Documentation/perf-list.txt
+++ b/tools/perf/Documentation/perf-list.txt
@@ -8,7 +8,8 @@ perf-list - List all symbolic event types
SYNOPSIS
--------
[verse]
-'perf list' [--no-desc] [--long-desc] [hw|sw|cache|tracepoint|pmu|sdt|event_glob]
+'perf list' [--no-desc] [--long-desc]
+ [hw|sw|cache|tracepoint|pmu|sdt|metric|metricgroup|event_glob]
DESCRIPTION
-----------
@@ -247,6 +248,10 @@ To limit the list use:
. 'sdt' to list all Statically Defined Tracepoint events.
+. 'metric' to list metrics
+
+. 'metricgroup' to list metricgroups with metrics.
+
. If none of the above is matched, it will apply the supplied glob to all
events, printing the ones that match.
diff --git a/tools/perf/builtin-list.c b/tools/perf/builtin-list.c
index 4bf2cb4d25aa..b2d2ad3dd478 100644
--- a/tools/perf/builtin-list.c
+++ b/tools/perf/builtin-list.c
@@ -15,6 +15,7 @@
#include "util/cache.h"
#include "util/pmu.h"
#include "util/debug.h"
+#include "util/metricgroup.h"
#include <subcmd/parse-options.h>
static bool desc_flag = true;
@@ -79,6 +80,10 @@ int cmd_list(int argc, const char **argv)
long_desc_flag, details_flag);
else if (strcmp(argv[i], "sdt") == 0)
print_sdt_events(NULL, NULL, raw_dump);
+ else if (strcmp(argv[i], "metric") == 0)
+ metricgroup__print(true, false, NULL, raw_dump);
+ else if (strcmp(argv[i], "metricgroup") == 0)
+ metricgroup__print(false, true, NULL, raw_dump);
else if ((sep = strchr(argv[i], ':')) != NULL) {
int sep_idx;
@@ -96,6 +101,7 @@ int cmd_list(int argc, const char **argv)
s[sep_idx] = '\0';
print_tracepoint_events(s, s + sep_idx + 1, raw_dump);
print_sdt_events(s, s + sep_idx + 1, raw_dump);
+ metricgroup__print(true, true, s, raw_dump);
free(s);
} else {
if (asprintf(&s, "*%s*", argv[i]) < 0) {
@@ -112,6 +118,7 @@ int cmd_list(int argc, const char **argv)
details_flag);
print_tracepoint_events(NULL, s, raw_dump);
print_sdt_events(NULL, s, raw_dump);
+ metricgroup__print(true, true, NULL, raw_dump);
free(s);
}
}
diff --git a/tools/perf/util/metricgroup.c b/tools/perf/util/metricgroup.c
index 7516b1746594..2d60114f1870 100644
--- a/tools/perf/util/metricgroup.c
+++ b/tools/perf/util/metricgroup.c
@@ -189,6 +189,182 @@ static bool match_metric(const char *n, const char *list)
return false;
}
+struct mep {
+ struct rb_node nd;
+ const char *name;
+ struct strlist *metrics;
+};
+
+static int mep_cmp(struct rb_node *rb_node, const void *entry)
+{
+ struct mep *a = container_of(rb_node, struct mep, nd);
+ struct mep *b = (struct mep *)entry;
+
+ return strcmp(a->name, b->name);
+}
+
+static struct rb_node *mep_new(struct rblist *rl __maybe_unused,
+ const void *entry)
+{
+ struct mep *me = malloc(sizeof(struct mep));
+
+ if (!me)
+ return NULL;
+ memcpy(me, entry, sizeof(struct mep));
+ me->name = strdup(me->name);
+ if (!me->name)
+ goto out_me;
+ me->metrics = strlist__new(NULL, NULL);
+ if (!me->metrics)
+ goto out_name;
+ return &me->nd;
+out_name:
+ free((char *)me->name);
+out_me:
+ free(me);
+ return NULL;
+}
+
+static struct mep *mep_lookup(struct rblist *groups, const char *name)
+{
+ struct rb_node *nd;
+ struct mep me = {
+ .name = name
+ };
+ nd = rblist__find(groups, &me);
+ if (nd)
+ return container_of(nd, struct mep, nd);
+ rblist__add_node(groups, &me);
+ nd = rblist__find(groups, &me);
+ if (nd)
+ return container_of(nd, struct mep, nd);
+ return NULL;
+}
+
+static void mep_delete(struct rblist *rl __maybe_unused,
+ struct rb_node *nd)
+{
+ struct mep *me = container_of(nd, struct mep, nd);
+
+ strlist__delete(me->metrics);
+ free((void *)me->name);
+ free(me);
+}
+
+static void metricgroup__print_strlist(struct strlist *metrics, bool raw)
+{
+ struct str_node *sn;
+ int n = 0;
+
+ strlist__for_each_entry (sn, metrics) {
+ if (raw)
+ printf("%s%s", n > 0 ? " " : "", sn->s);
+ else
+ printf(" %s\n", sn->s);
+ n++;
+ }
+ if (raw)
+ putchar('\n');
+}
+
+void metricgroup__print(bool metrics, bool metricgroups, char *filter,
+ bool raw)
+{
+ struct pmu_events_map *map = perf_pmu__find_map();
+ struct pmu_event *pe;
+ int i;
+ struct rblist groups;
+ struct rb_node *node, *next;
+ struct strlist *metriclist = NULL;
+
+ if (!map)
+ return;
+
+ if (!metricgroups) {
+ metriclist = strlist__new(NULL, NULL);
+ if (!metriclist)
+ return;
+ }
+
+ rblist__init(&groups);
+ groups.node_new = mep_new;
+ groups.node_cmp = mep_cmp;
+ groups.node_delete = mep_delete;
+ for (i = 0; ; i++) {
+ const char *g;
+ pe = &map->table[i];
+
+ if (!pe->name && !pe->metric_group && !pe->metric_name)
+ break;
+ if (!pe->metric_expr)
+ continue;
+ g = pe->metric_group;
+ if (!g && pe->metric_name) {
+ if (pe->name)
+ continue;
+ g = "No_group";
+ }
+ if (g) {
+ char *omg;
+ char *mg = strdup(g);
+
+ if (!mg)
+ return;
+ omg = mg;
+ while ((g = strsep(&mg, ";")) != NULL) {
+ struct mep *me;
+ char *s;
+
+ if (*g == 0)
+ g = "No_group";
+ while (isspace(*g))
+ g++;
+ if (filter && !strstr(g, filter))
+ continue;
+ if (raw)
+ s = (char *)pe->metric_name;
+ else {
+ if (asprintf(&s, "%s\n\t[%s]",
+ pe->metric_name, pe->desc) < 0)
+ return;
+ }
+
+ if (!s)
+ continue;
+
+ if (!metricgroups) {
+ strlist__add(metriclist, s);
+ } else {
+ me = mep_lookup(&groups, g);
+ if (!me)
+ continue;
+ strlist__add(me->metrics, s);
+ }
+ }
+ free(omg);
+ }
+ }
+
+ if (metricgroups && !raw)
+ printf("\nMetric Groups:\n\n");
+ else if (metrics && !raw)
+ printf("\nMetrics:\n\n");
+
+ for (node = rb_first(&groups.entries); node; node = next) {
+ struct mep *me = container_of(node, struct mep, nd);
+
+ if (metricgroups)
+ printf("%s%s%s", me->name, metrics ? ":" : "", raw ? " " : "\n");
+ if (metrics)
+ metricgroup__print_strlist(me->metrics, raw);
+ next = rb_next(node);
+ rblist__remove_node(&groups, node);
+ }
+ if (!metricgroups)
+ metricgroup__print_strlist(metriclist, raw);
+ strlist__delete(metriclist);
+}
+
static int metricgroup__add_metric(const char *metric, struct strbuf *events,
struct list_head *group_list)
{
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index cd80c4eac569..97507d5c37dc 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -28,6 +28,7 @@
#include "probe-file.h"
#include "asm/bug.h"
#include "util/parse-branch-options.h"
+#include "metricgroup.h"
#define MAX_NAME_LEN 100
@@ -2372,6 +2373,8 @@ void print_events(const char *event_glob, bool name_only, bool quiet_flag,
print_tracepoint_events(NULL, NULL, name_only);
print_sdt_events(NULL, NULL, name_only);
+
+ metricgroup__print(true, true, NULL, name_only);
}
int parse_events__is_hardcoded_term(struct parse_events_term *term)
--
2.9.5
[toc] | [prev] | [next] | [standalone]
| From | Andi Kleen <andi@firstfloor.org> |
|---|---|
| Date | 2017-08-31 21:50 +0200 |
| Subject | [PATCH v3 11/11] perf, tools, stat: Update walltime_nsecs_stats in interval mode |
| Message-ID | <ukDDs-1ub-27@gated-at.bofh.it> |
| In reply to | #1724527 |
From: Andi Kleen <ak@linux.intel.com>
Some metrics (like GFLOPs) need walltime_nsecs_stats for each interval.
Compute it for each interval instead of only at the end.
Pointed out by Jiri.
Signed-off-by: Andi Kleen <ak@linux.intel.com>
---
tools/perf/builtin-stat.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/tools/perf/builtin-stat.c b/tools/perf/builtin-stat.c
index bf6ae0144ecf..60206a1e03aa 100644
--- a/tools/perf/builtin-stat.c
+++ b/tools/perf/builtin-stat.c
@@ -415,6 +415,8 @@ static void process_interval(void)
pr_err("failed to write stat round event\n");
}
+ init_stats(&walltime_nsecs_stats);
+ update_stats(&walltime_nsecs_stats, stat_config.interval * 1000000);
print_counters(&rs, 0, NULL);
}
--
2.9.5
[toc] | [prev] | [next] | [standalone]
| From | Andi Kleen <andi@firstfloor.org> |
|---|---|
| Date | 2017-08-31 21:50 +0200 |
| Subject | [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat |
| Message-ID | <ukDDs-1ub-29@gated-at.bofh.it> |
| In reply to | #1724527 |
From: Andi Kleen <ak@linux.intel.com>
Add generic support for standalone metrics specified in JSON files
to perf stat. A metric is a formula that uses multiple events
to compute a higher level result (e.g. IPC).
Previously metrics were always tied to an event and automatically
enabled with that event. But now change it that we can have
standalone metrics. They are in the same JSON data structure
as events, but don't have an event name.
We also allow to organize the metrics in metric groups, which
allows a short cut to select several related metrics at once.
Add a new -M / --metrics option to perf stat that adds the metrics
or metric groups specified.
Add the core code to manage and parse the metric groups. They
are collected from the JSON data structures into a separate rblist.
When computing shadow values look for metrics in that list.
Then they are computed using the existing saved values infrastructure
in stat-shadow.c
The actual JSON metrics are in a separate pull request.
% perf stat -M Summary --metric-only -a sleep 1
Performance counter stats for 'system wide':
Instructions CLKS CPU_Utilization GFLOPs SMT_2T_Utilization Kernel_Utilization
317614222.0 1392930775.0 0.0 0.0 0.2 0.1
1.001497549 seconds time elapsed
% perf stat -M GFLOPs flops
Performance counter stats for 'flops':
3,999,541,471 fp_comp_ops_exe.sse_scalar_single # 1.2 GFLOPs (66.65%)
14 fp_comp_ops_exe.sse_scalar_double (66.65%)
0 fp_comp_ops_exe.sse_packed_double (66.67%)
0 fp_comp_ops_exe.sse_packed_single (66.70%)
0 simd_fp_256.packed_double (66.70%)
0 simd_fp_256.packed_single (66.67%)
0 duration_time
3.238372845 seconds time elapsed
v2: Add missing header file
v3: Move find_map to pmu.c
Signed-off-by: Andi Kleen <ak@linux.intel.com>
---
tools/perf/Documentation/perf-stat.txt | 7 +
tools/perf/builtin-stat.c | 18 +-
tools/perf/util/Build | 1 +
tools/perf/util/metricgroup.c | 313 +++++++++++++++++++++++++++++++++
tools/perf/util/metricgroup.h | 31 ++++
tools/perf/util/pmu.c | 5 +-
tools/perf/util/stat-shadow.c | 22 ++-
tools/perf/util/stat.h | 4 +-
8 files changed, 395 insertions(+), 6 deletions(-)
create mode 100644 tools/perf/util/metricgroup.c
create mode 100644 tools/perf/util/metricgroup.h
diff --git a/tools/perf/Documentation/perf-stat.txt b/tools/perf/Documentation/perf-stat.txt
index c37d61682dfb..823fce7674bb 100644
--- a/tools/perf/Documentation/perf-stat.txt
+++ b/tools/perf/Documentation/perf-stat.txt
@@ -199,6 +199,13 @@ Aggregate counts per processor socket for system-wide mode measurements.
--per-core::
Aggregate counts per physical processor for system-wide mode measurements.
+-M::
+--metrics::
+Print metrics or metricgroups specified in a comma separated list.
+For a group all metrics from the group are added.
+The events from the metrics are automatically measured.
+See perf list output for the possble metrics and metricgroups.
+
-A::
--no-aggr::
Do not aggregate counts across all monitored CPUs.
diff --git a/tools/perf/builtin-stat.c b/tools/perf/builtin-stat.c
index 501c1e0272fe..045efd7d7785 100644
--- a/tools/perf/builtin-stat.c
+++ b/tools/perf/builtin-stat.c
@@ -65,6 +65,7 @@
#include "util/tool.h"
#include "util/group.h"
#include "util/string2.h"
+#include "util/metricgroup.h"
#include "asm/bug.h"
#include <linux/time64.h>
@@ -133,6 +134,8 @@ static const char *smi_cost_attrs = {
static struct perf_evlist *evsel_list;
+static struct rblist metric_events;
+
static struct target target = {
.uid = UINT_MAX,
};
@@ -1234,7 +1237,7 @@ static void printout(int id, int nr, struct perf_evsel *counter, double uval,
perf_stat__print_shadow_stats(counter, uval,
first_shadow_cpu(counter, id),
- &out);
+ &out, &metric_events);
if (!csv_output && !metric_only) {
print_noise(counter, noise);
print_running(run, ena);
@@ -1565,7 +1568,8 @@ static void print_metric_headers(const char *prefix, bool no_indent)
os.evsel = counter;
perf_stat__print_shadow_stats(counter, 0,
0,
- &out);
+ &out,
+ &metric_events);
}
fputc('\n', stat_config.output);
}
@@ -1789,6 +1793,13 @@ static int enable_metric_only(const struct option *opt __maybe_unused,
return 0;
}
+static int parse_metric_groups(const struct option *opt,
+ const char *str,
+ int unset __maybe_unused)
+{
+ return metricgroup__parse_groups(opt, str, &metric_events);
+}
+
static const struct option stat_options[] = {
OPT_BOOLEAN('T', "transaction", &transaction_run,
"hardware transaction statistics"),
@@ -1854,6 +1865,9 @@ static const struct option stat_options[] = {
"measure topdown level 1 statistics"),
OPT_BOOLEAN(0, "smi-cost", &smi_cost,
"measure SMI cost"),
+ OPT_CALLBACK('M', "metrics", &evsel_list, "metric/metric group list",
+ "monitor specified metrics or metric groups (separated by ,)",
+ parse_metric_groups),
OPT_END()
};
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index 94518c1bf8b6..71ab8466714d 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -34,6 +34,7 @@ libperf-y += dso.o
libperf-y += symbol.o
libperf-y += symbol_fprintf.o
libperf-y += color.o
+libperf-y += metricgroup.o
libperf-y += header.o
libperf-y += callchain.o
libperf-y += values.o
diff --git a/tools/perf/util/metricgroup.c b/tools/perf/util/metricgroup.c
new file mode 100644
index 000000000000..7516b1746594
--- /dev/null
+++ b/tools/perf/util/metricgroup.c
@@ -0,0 +1,313 @@
+/*
+ * Copyright (c) 2017, Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ */
+
+/* Manage metrics and groups of metrics from JSON files */
+
+#include "metricgroup.h"
+#include "evlist.h"
+#include "strbuf.h"
+#include "pmu.h"
+#include "expr.h"
+#include "rblist.h"
+#include "pmu.h"
+#include <string.h>
+#include <stdbool.h>
+#include <errno.h>
+#include "pmu-events/pmu-events.h"
+#include "strbuf.h"
+#include "strlist.h"
+#include <assert.h>
+#include <ctype.h>
+
+struct metric_event *metricgroup__lookup(struct rblist *metric_events,
+ struct perf_evsel *evsel,
+ bool create)
+{
+ struct rb_node *nd;
+ struct metric_event me = {
+ .evsel = evsel
+ };
+ nd = rblist__find(metric_events, &me);
+ if (nd)
+ return container_of(nd, struct metric_event, nd);
+ if (create) {
+ rblist__add_node(metric_events, &me);
+ nd = rblist__find(metric_events, &me);
+ if (nd)
+ return container_of(nd, struct metric_event, nd);
+ }
+ return NULL;
+}
+
+static int metric_event_cmp(struct rb_node *rb_node, const void *entry)
+{
+ struct metric_event *a = container_of(rb_node,
+ struct metric_event,
+ nd);
+ const struct metric_event *b = entry;
+
+ if (a->evsel == b->evsel)
+ return 0;
+ if ((char *)a->evsel < (char *)b->evsel)
+ return -1;
+ return +1;
+}
+
+static struct rb_node *metric_event_new(struct rblist *rblist __maybe_unused,
+ const void *entry)
+{
+ struct metric_event *me = malloc(sizeof(struct metric_event));
+
+ if (!me)
+ return NULL;
+ memcpy(me, entry, sizeof(struct metric_event));
+ me->evsel = ((struct metric_event *)entry)->evsel;
+ INIT_LIST_HEAD(&me->head);
+ return &me->nd;
+}
+
+static void metricgroup__rblist_init(struct rblist *metric_events)
+{
+ rblist__init(metric_events);
+ metric_events->node_cmp = metric_event_cmp;
+ metric_events->node_new = metric_event_new;
+}
+
+struct egroup {
+ struct list_head nd;
+ int idnum;
+ const char **ids;
+ const char *metric_name;
+ const char *metric_expr;
+};
+
+static struct perf_evsel *find_evsel(struct perf_evlist *perf_evlist,
+ const char **ids,
+ int idnum,
+ struct perf_evsel **metric_events)
+{
+ struct perf_evsel *ev, *start = NULL;
+ int ind = 0;
+
+ evlist__for_each_entry (perf_evlist, ev) {
+ if (!strcmp(ev->name, ids[ind])) {
+ metric_events[ind] = ev;
+ if (ind == 0)
+ start = ev;
+ if (++ind == idnum) {
+ metric_events[ind] = NULL;
+ return start;
+ }
+ } else {
+ ind = 0;
+ start = NULL;
+ }
+ }
+ /*
+ * This can happen when an alias expands to multiple
+ * events, like for uncore events.
+ * We don't support this case for now.
+ */
+ return NULL;
+}
+
+static int metricgroup__setup_events(struct list_head *groups,
+ struct perf_evlist *perf_evlist,
+ struct rblist *metric_events_list)
+{
+ struct metric_event *me;
+ struct metric_expr *expr;
+ int i = 0;
+ int ret = 0;
+ struct egroup *eg;
+ struct perf_evsel *evsel;
+
+ list_for_each_entry (eg, groups, nd) {
+ struct perf_evsel **metric_events;
+
+ metric_events = calloc(sizeof(void *), eg->idnum + 1);
+ if (!metric_events) {
+ ret = -ENOMEM;
+ break;
+ }
+ evsel = find_evsel(perf_evlist, eg->ids, eg->idnum,
+ metric_events);
+ if (!evsel) {
+ pr_debug("Cannot resolve %s: %s\n",
+ eg->metric_name, eg->metric_expr);
+ continue;
+ }
+ for (i = 0; i < eg->idnum; i++)
+ metric_events[i]->collect_stat = true;
+ me = metricgroup__lookup(metric_events_list, evsel, true);
+ if (!me) {
+ ret = -ENOMEM;
+ break;
+ }
+ expr = malloc(sizeof(struct metric_expr));
+ if (!expr) {
+ ret = -ENOMEM;
+ break;
+ }
+ expr->metric_expr = eg->metric_expr;
+ expr->metric_name = eg->metric_name;
+ expr->metric_events = metric_events;
+ list_add(&expr->nd, &me->head);
+ }
+ return ret;
+}
+
+static bool match_metric(const char *n, const char *list)
+{
+ int len;
+ char *m;
+
+ if (!list)
+ return false;
+ if (!strcmp(list, "all"))
+ return true;
+ if (!n)
+ return !strcasecmp(list, "No_group");
+ len = strlen(list);
+ m = strcasestr(n, list);
+ if (!m)
+ return false;
+ if ((m == n || m[-1] == ';' || m[-1] == ' ') &&
+ (m[len] == 0 || m[len] == ';'))
+ return true;
+ return false;
+}
+
+static int metricgroup__add_metric(const char *metric, struct strbuf *events,
+ struct list_head *group_list)
+{
+ struct pmu_events_map *map = perf_pmu__find_map();
+ struct pmu_event *pe;
+ int ret = -EINVAL;
+ int i, j;
+
+ strbuf_init(events, 100);
+ strbuf_addf(events, "%s", "");
+
+ if (!map)
+ return 0;
+
+ for (i = 0; ; i++) {
+ pe = &map->table[i];
+
+ if (!pe->name && !pe->metric_group && !pe->metric_name)
+ break;
+ if (!pe->metric_expr)
+ continue;
+ if (match_metric(pe->metric_group, metric) ||
+ match_metric(pe->metric_name, metric)) {
+ const char **ids;
+ int idnum;
+ struct egroup *eg;
+
+ pr_debug("metric expr %s for %s\n", pe->metric_expr, pe->metric_name);
+
+ if (expr__find_other(pe->metric_expr,
+ NULL, &ids, &idnum) < 0)
+ continue;
+ if (events->len > 0)
+ strbuf_addf(events, ",");
+ for (j = 0; j < idnum; j++) {
+ pr_debug("found event %s\n", ids[j]);
+ strbuf_addf(events, "%s%s",
+ j == 0 ? "{" : ",",
+ ids[j]);
+ }
+ strbuf_addf(events, "}:W");
+
+ eg = malloc(sizeof(struct egroup));
+ if (!eg) {
+ ret = -ENOMEM;
+ break;
+ }
+ eg->ids = ids;
+ eg->idnum = idnum;
+ eg->metric_name = pe->metric_name;
+ eg->metric_expr = pe->metric_expr;
+ list_add_tail(&eg->nd, group_list);
+ ret = 0;
+ }
+ }
+ return ret;
+}
+
+static int metricgroup__add_metric_list(const char *list, struct strbuf *events,
+ struct list_head *group_list)
+{
+ char *llist, *nlist, *p;
+ int ret = -EINVAL;
+
+ nlist = strdup(list);
+ if (!nlist)
+ return -ENOMEM;
+ llist = nlist;
+ while ((p = strsep(&llist, ",")) != NULL) {
+ ret = metricgroup__add_metric(p, events, group_list);
+ if (ret == -EINVAL) {
+ fprintf(stderr, "Cannot find metric or group `%s'\n",
+ p);
+ break;
+ }
+ }
+ free(nlist);
+ return ret;
+}
+
+static void metricgroup__free_egroups(struct list_head *group_list)
+{
+ struct egroup *eg, *egtmp;
+ int i;
+
+ list_for_each_entry_safe (eg, egtmp, group_list, nd) {
+ for (i = 0; i < eg->idnum; i++)
+ free((char *)eg->ids[i]);
+ free(eg->ids);
+ free(eg);
+ }
+}
+
+int metricgroup__parse_groups(const struct option *opt,
+ const char *str,
+ struct rblist *metric_events)
+{
+ struct parse_events_error parse_error;
+ struct perf_evlist *perf_evlist = *(struct perf_evlist **)opt->value;
+ struct strbuf extra_events;
+ LIST_HEAD(group_list);
+ int ret;
+
+ if (metric_events->nr_entries == 0)
+ metricgroup__rblist_init(metric_events);
+ ret = metricgroup__add_metric_list(str, &extra_events, &group_list);
+ if (ret)
+ return ret;
+ pr_debug("adding %s\n", extra_events.buf);
+ memset(&parse_error, 0, sizeof(struct parse_events_error));
+ ret = parse_events(perf_evlist, extra_events.buf, &parse_error);
+ if (ret) {
+ pr_err("Cannot set up events %s\n", extra_events.buf);
+ goto out;
+ }
+ strbuf_release(&extra_events);
+ ret = metricgroup__setup_events(&group_list, perf_evlist,
+ metric_events);
+out:
+ metricgroup__free_egroups(&group_list);
+ return ret;
+}
diff --git a/tools/perf/util/metricgroup.h b/tools/perf/util/metricgroup.h
new file mode 100644
index 000000000000..06854e125ee7
--- /dev/null
+++ b/tools/perf/util/metricgroup.h
@@ -0,0 +1,31 @@
+#ifndef METRICGROUP_H
+#define METRICGROUP_H 1
+
+#include "linux/list.h"
+#include "rblist.h"
+#include <subcmd/parse-options.h>
+#include "evlist.h"
+#include "strbuf.h"
+
+struct metric_event {
+ struct rb_node nd;
+ struct perf_evsel *evsel;
+ struct list_head head; /* list of metric_expr */
+};
+
+struct metric_expr {
+ struct list_head nd;
+ const char *metric_expr;
+ const char *metric_name;
+ struct perf_evsel **metric_events;
+};
+
+struct metric_event *metricgroup__lookup(struct rblist *metric_events,
+ struct perf_evsel *evsel,
+ bool create);
+int metricgroup__parse_groups(const struct option *opt,
+ const char *str,
+ struct rblist *metric_events);
+
+void metricgroup__print(bool metrics, bool groups, char *filter, bool raw);
+#endif
diff --git a/tools/perf/util/pmu.c b/tools/perf/util/pmu.c
index ed25d7f88731..7070638ab600 100644
--- a/tools/perf/util/pmu.c
+++ b/tools/perf/util/pmu.c
@@ -580,8 +580,11 @@ static void pmu_add_cpu_aliases(struct list_head *head, const char *name)
const char *pname;
pe = &map->table[i++];
- if (!pe->name)
+ if (!pe->name) {
+ if (pe->metric_group || pe->metric_name)
+ continue;
break;
+ }
pname = pe->pmu ? pe->pmu : "cpu";
if (strncmp(pname, name, strlen(pname)))
diff --git a/tools/perf/util/stat-shadow.c b/tools/perf/util/stat-shadow.c
index 8c7ab29169b9..42e6c17be7ff 100644
--- a/tools/perf/util/stat-shadow.c
+++ b/tools/perf/util/stat-shadow.c
@@ -6,6 +6,7 @@
#include "rblist.h"
#include "evlist.h"
#include "expr.h"
+#include "metricgroup.h"
enum {
CTX_BIT_USER = 1 << 0,
@@ -671,13 +672,16 @@ static void generic_metric(const char *metric_expr,
void perf_stat__print_shadow_stats(struct perf_evsel *evsel,
double avg, int cpu,
- struct perf_stat_output_ctx *out)
+ struct perf_stat_output_ctx *out,
+ struct rblist *metric_events)
{
void *ctxp = out->ctx;
print_metric_t print_metric = out->print_metric;
double total, ratio = 0.0, total2;
const char *color = NULL;
int ctx = evsel_context(evsel);
+ struct metric_event *me;
+ int num = 1;
if (perf_evsel__match(evsel, HARDWARE, HW_INSTRUCTIONS)) {
total = avg_stats(&runtime_cycles_stats[ctx][cpu]);
@@ -880,6 +884,20 @@ void perf_stat__print_shadow_stats(struct perf_evsel *evsel,
} else if (perf_stat_evsel__is(evsel, SMI_NUM)) {
print_smi_cost(cpu, evsel, out);
} else {
- print_metric(ctxp, NULL, NULL, NULL, 0);
+ num = 0;
}
+
+ if ((me = metricgroup__lookup(metric_events, evsel, false)) != NULL) {
+ struct metric_expr *mexp;
+
+ list_for_each_entry (mexp, &me->head, nd) {
+ if (num++ > 0)
+ out->new_line(ctxp);
+ generic_metric(mexp->metric_expr, mexp->metric_events,
+ evsel->name, mexp->metric_name,
+ avg, cpu, ctx, out);
+ }
+ }
+ if (num == 0)
+ print_metric(ctxp, NULL, NULL, NULL, 0);
}
diff --git a/tools/perf/util/stat.h b/tools/perf/util/stat.h
index eacaf958e19d..47915df346fb 100644
--- a/tools/perf/util/stat.h
+++ b/tools/perf/util/stat.h
@@ -91,9 +91,11 @@ struct perf_stat_output_ctx {
bool force_header;
};
+struct rblist;
void perf_stat__print_shadow_stats(struct perf_evsel *evsel,
double avg, int cpu,
- struct perf_stat_output_ctx *out);
+ struct perf_stat_output_ctx *out,
+ struct rblist *metric_events);
void perf_stat__collect_metric_expr(struct perf_evlist *);
int perf_evlist__alloc_stats(struct perf_evlist *evlist, bool alloc_raw);
--
2.9.5
[toc] | [prev] | [next] | [standalone]
| From | Arnaldo Carvalho de Melo <acme@kernel.org> |
|---|---|
| Date | 2017-09-04 19:20 +0200 |
| Subject | Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat |
| Message-ID | <um3ct-8hZ-15@gated-at.bofh.it> |
| In reply to | #1724531 |
Em Thu, Aug 31, 2017 at 12:40:31PM -0700, Andi Kleen escreveu:
> The actual JSON metrics are in a separate pull request.
Yeah, I noticed when trying to test it :-\
Was this pull req submitted?
- Arnaldo
> % perf stat -M Summary --metric-only -a sleep 1
[root@jouet ~]# perf stat -M Summary --metric-only -a sleep 1
Cannot find metric or group `Summary'
Usage: perf stat [<options>] [<command>]
-M, --metrics <metric/metric group list>
monitor specified metrics or metric groups (separated by ,)
[root@jouet ~]# perf stat -M GFLOPs flops
Cannot find metric or group `GFLOPs'
Usage: perf stat [<options>] [<command>]
-M, --metrics <metric/metric group list>
monitor specified metrics or metric groups (separated by ,)
[root@jouet ~]#
[toc] | [prev] | [next] | [standalone]
| From | Andi Kleen <andi@firstfloor.org> |
|---|---|
| Date | 2017-09-04 19:40 +0200 |
| Subject | Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat |
| Message-ID | <um3vP-8os-7@gated-at.bofh.it> |
| In reply to | #1726197 |
On Mon, Sep 04, 2017 at 02:11:28PM -0300, Arnaldo Carvalho de Melo wrote: > Em Thu, Aug 31, 2017 at 12:40:31PM -0700, Andi Kleen escreveu: > > The actual JSON metrics are in a separate pull request. > > Yeah, I noticed when trying to test it :-\ > > Was this pull req submitted? Not yet. I was waiting to finish review. It makes perf crash until you apply the metricgroup patchkit first. But you can get it here git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-misc.git perf/intel-json-metrics-2 -Andi
[toc] | [prev] | [next] | [standalone]
| From | Arnaldo Carvalho de Melo <acme@kernel.org> |
|---|---|
| Date | 2017-09-05 20:10 +0200 |
| Subject | Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat |
| Message-ID | <umqsp-6ff-3@gated-at.bofh.it> |
| In reply to | #1726208 |
Em Mon, Sep 04, 2017 at 10:37:25AM -0700, Andi Kleen escreveu:
> On Mon, Sep 04, 2017 at 02:11:28PM -0300, Arnaldo Carvalho de Melo wrote:
> > Em Thu, Aug 31, 2017 at 12:40:31PM -0700, Andi Kleen escreveu:
> > > The actual JSON metrics are in a separate pull request.
> >
> > Yeah, I noticed when trying to test it :-\
> >
> > Was this pull req submitted?
>
> Not yet. I was waiting to finish review.
>
> It makes perf crash until you apply the metricgroup patchkit first.
>
> But you can get it here
>
> git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-misc.git perf/intel-json-metrics-2
Ok, so I tried installing the Broadwell ones (one of my test machines)
and got into:
[root@jouet ~]# perf stat -M Pipeline flops
bpf: builtin compilation failed: -95, try external compiler
ERROR: problems with path cpu/uops_executed.c: No such file or directory
Cannot set up events {uops_retired.retire_slots,inst_retired.any}:W,{inst_retired.any,cycles}:W,{uops_executed.thread,cpu/uops_executed.core,cmask=1/,uops_executed.cycles_ge_1_uop_exec}:W
Usage: perf stat [<options>] [<command>]
-M, --metrics <metric/metric group list>
monitor specified metrics or metric groups (separated by ,)
[root@jouet ~]# perf stat -M Pipeline flops
Wasn't this already fixed?
This is with these patches applied:
[acme@jouet linux]$ git log --oneline -5
5e6bf24a1929 (HEAD) perf stat: Support JSON metrics in perf stat
b1e1f50913c8 perf pmu: Extract function to get JSON alias map
af8aca46b528 perf stat: Print generic metric header even for failed expressions
15669491ee54 perf stat: Factor out generic metric printing
7d8900f2984c perf vendor events: Support metric_group and no event name in JSON parser
[acme@jouet linux]$ git log --oneline -10
5e6bf24a1929 (HEAD) perf stat: Support JSON metrics in perf stat
b1e1f50913c8 perf pmu: Extract function to get JSON alias map
af8aca46b528 perf stat: Print generic metric header even for failed expressions
15669491ee54 perf stat: Factor out generic metric printing
7d8900f2984c perf vendor events: Support metric_group and no event name in JSON parser
8f6ea3aac2e2 perf tools: Support weak groups in 'perf stat'
b5ef5ea5e8ed perf sched timehist: Add pid and tid options
eba9fac01761 (tag: perf-core-for-mingo-4.14-20170901, acme/perf/core) perf annotate browser: Help for cycling thru hottest instructions with TAB/shift+TAB
63ce8449bc10 perf stat: Only auto-merge events that are PMU aliases
fc33dccba395 perf test: Add test case for PERF_SAMPLE_PHYS_ADDR
[acme@jouet linux]$
Will try with all of them...
- Arnaldo
[toc] | [prev] | [next] | [standalone]
| From | Arnaldo Carvalho de Melo <acme@kernel.org> |
|---|---|
| Date | 2017-09-05 20:20 +0200 |
| Subject | Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat |
| Message-ID | <umqC7-6kM-17@gated-at.bofh.it> |
| In reply to | #1726909 |
Em Tue, Sep 05, 2017 at 03:09:19PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Mon, Sep 04, 2017 at 10:37:25AM -0700, Andi Kleen escreveu:
> > On Mon, Sep 04, 2017 at 02:11:28PM -0300, Arnaldo Carvalho de Melo wrote:
> > > Em Thu, Aug 31, 2017 at 12:40:31PM -0700, Andi Kleen escreveu:
> > > > The actual JSON metrics are in a separate pull request.
> > >
> > > Yeah, I noticed when trying to test it :-\
> > >
> > > Was this pull req submitted?
> >
> > Not yet. I was waiting to finish review.
> >
> > It makes perf crash until you apply the metricgroup patchkit first.
> >
> > But you can get it here
> >
> > git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-misc.git perf/intel-json-metrics-2
>
> Ok, so I tried installing the Broadwell ones (one of my test machines)
> and got into:
>
> [root@jouet ~]# perf stat -M Pipeline flops
> bpf: builtin compilation failed: -95, try external compiler
So there is no 'flops' binary here, unsure if the problem is related to
that, if so, we need a fix.
But that doesn't seem to be the case, as with a metric that works I get
a sensible error message:
[root@jouet ~]# perf stat -M Memory_BW --metric-only -a flops
Workload failed: No such file or directory
[root@jouet ~]#
Anyway, testing with some other metric I got some results:
[root@jouet ~]# perf stat -M FLOPS --metric-only -a sleep 1
Cannot set up events {fp_arith_inst_retired.scalar_single,fp_arith_inst_retired.scalar_double,fp_arith_inst_retired.128b_packed_double,fp_arith_inst_retired.128b_packed_single,fp_arith_inst_retired.256b_packed_double,fp_arith_inst_retired.256b_packed_single,duration_time}:W
Usage: perf stat [<options>] [<command>]
-M, --metrics <metric/metric group list>
monitor specified metrics or metric groups (separated by ,)
[root@jouet ~]# perf stat -M Memory_BW --metric-only -a sleep 1
Performance counter stats for 'system wide':
MLP
2.9
1.001544615 seconds time elapsed
[root@jouet ~]#
> ERROR: problems with path cpu/uops_executed.c: No such file or directory
> Cannot set up events {uops_retired.retire_slots,inst_retired.any}:W,{inst_retired.any,cycles}:W,{uops_executed.thread,cpu/uops_executed.core,cmask=1/,uops_executed.cycles_ge_1_uop_exec}:W
>
> Usage: perf stat [<options>] [<command>]
>
> -M, --metrics <metric/metric group list>
> monitor specified metrics or metric groups (separated by ,)
> [root@jouet ~]# perf stat -M Pipeline flops
>
>
> Wasn't this already fixed?
>
> This is with these patches applied:
>
> [acme@jouet linux]$ git log --oneline -5
> 5e6bf24a1929 (HEAD) perf stat: Support JSON metrics in perf stat
> b1e1f50913c8 perf pmu: Extract function to get JSON alias map
> af8aca46b528 perf stat: Print generic metric header even for failed expressions
> 15669491ee54 perf stat: Factor out generic metric printing
> 7d8900f2984c perf vendor events: Support metric_group and no event name in JSON parser
> [acme@jouet linux]$ git log --oneline -10
> 5e6bf24a1929 (HEAD) perf stat: Support JSON metrics in perf stat
> b1e1f50913c8 perf pmu: Extract function to get JSON alias map
> af8aca46b528 perf stat: Print generic metric header even for failed expressions
> 15669491ee54 perf stat: Factor out generic metric printing
> 7d8900f2984c perf vendor events: Support metric_group and no event name in JSON parser
> 8f6ea3aac2e2 perf tools: Support weak groups in 'perf stat'
> b5ef5ea5e8ed perf sched timehist: Add pid and tid options
> eba9fac01761 (tag: perf-core-for-mingo-4.14-20170901, acme/perf/core) perf annotate browser: Help for cycling thru hottest instructions with TAB/shift+TAB
> 63ce8449bc10 perf stat: Only auto-merge events that are PMU aliases
> fc33dccba395 perf test: Add test case for PERF_SAMPLE_PHYS_ADDR
> [acme@jouet linux]$
>
> Will try with all of them...
>
> - Arnaldo
[toc] | [prev] | [next] | [standalone]
| From | Arnaldo Carvalho de Melo <acme@kernel.org> |
|---|---|
| Date | 2017-09-05 20:40 +0200 |
| Subject | Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat |
| Message-ID | <umqVs-6sg-17@gated-at.bofh.it> |
| In reply to | #1726916 |
Em Tue, Sep 05, 2017 at 03:16:12PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Tue, Sep 05, 2017 at 03:09:19PM -0300, Arnaldo Carvalho de Melo escreveu:
> > Em Mon, Sep 04, 2017 at 10:37:25AM -0700, Andi Kleen escreveu:
> > > On Mon, Sep 04, 2017 at 02:11:28PM -0300, Arnaldo Carvalho de Melo wrote:
> > > > Em Thu, Aug 31, 2017 at 12:40:31PM -0700, Andi Kleen escreveu:
> > > > > The actual JSON metrics are in a separate pull request.
> > > >
> > > > Yeah, I noticed when trying to test it :-\
> > > >
> > > > Was this pull req submitted?
> > >
> > > Not yet. I was waiting to finish review.
> > >
> > > It makes perf crash until you apply the metricgroup patchkit first.
> > >
> > > But you can get it here
> > >
> > > git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-misc.git perf/intel-json-metrics-2
> >
> > Ok, so I tried installing the Broadwell ones (one of my test machines)
> > and got into:
> >
> > [root@jouet ~]# perf stat -M Pipeline flops
> > bpf: builtin compilation failed: -95, try external compiler
further info:
[acme@jouet linux]$ perf stat -v -M GFLOPs sleep 1
Using CPUID GenuineIntel-6-3D
metric expr ( 1*( fp_arith_inst_retired.scalar_single + fp_arith_inst_retired.scalar_double ) + 2* fp_arith_inst_retired.128b_packed_double + 4*( fp_arith_inst_retired.128b_packed_single + fp_arith_inst_retired.256b_packed_double ) + 8* fp_arith_inst_retired.256b_packed_single ) / 1000000000 / duration_time for GFLOPs
found event fp_arith_inst_retired.scalar_single
found event fp_arith_inst_retired.scalar_double
found event fp_arith_inst_retired.128b_packed_double
found event fp_arith_inst_retired.128b_packed_single
found event fp_arith_inst_retired.256b_packed_double
found event fp_arith_inst_retired.256b_packed_single
found event duration_time
adding {fp_arith_inst_retired.scalar_single,fp_arith_inst_retired.scalar_double,fp_arith_inst_retired.128b_packed_double,fp_arith_inst_retired.128b_packed_single,fp_arith_inst_retired.256b_packed_double,fp_arith_inst_retired.256b_packed_single,duration_time}:W
fp_arith_inst_retired.scalar_single -> cpu/umask=0x2,period=2000003,event=0xc7/
fp_arith_inst_retired.scalar_double -> cpu/umask=0x1,period=2000003,event=0xc7/
fp_arith_inst_retired.128b_packed_double -> cpu/umask=0x4,period=2000003,event=0xc7/
fp_arith_inst_retired.128b_packed_single -> cpu/umask=0x8,period=2000003,event=0xc7/
fp_arith_inst_retired.256b_packed_double -> cpu/umask=0x10,period=2000003,event=0xc7/
fp_arith_inst_retired.256b_packed_single -> cpu/umask=0x20,period=2000003,event=0xc7/
Cannot set up events {fp_arith_inst_retired.scalar_single,fp_arith_inst_retired.scalar_double,fp_arith_inst_retired.128b_packed_double,fp_arith_inst_retired.128b_packed_single,fp_arith_inst_retired.256b_packed_double,fp_arith_inst_retired.256b_packed_single,duration_time}:W
Usage: perf stat [<options>] [<command>]
-M, --metrics <metric/metric group list>
monitor specified metrics or metric groups (separated by ,)
[acme@jouet linux]$
Humm, so it uses that duration_time and that gets added only later?
/me tries with all the patches:
Yeah:
[acme@jouet linux]$ perf stat -M GFLOPs sleep 1
Performance counter stats for 'sleep 1':
0 fp_arith_inst_retired.scalar_single:u
3 fp_arith_inst_retired.scalar_double:u
0 fp_arith_inst_retired.128b_packed_double:u
0 fp_arith_inst_retired.128b_packed_single:u
0 fp_arith_inst_retired.256b_packed_double:u (40.86%)
<not counted> fp_arith_inst_retired.256b_packed_single:u (0.00%)
0 duration_time:u
1.003398909 seconds time elapsed
[acme@jouet linux]$
With --metric-only I get nothing, what am I doing wrong?
[acme@jouet linux]$ perf stat -M GFLOPs --metric-only sleep 1
Performance counter stats for 'sleep 1':
GFLOPs
1.003217768 seconds time elapsed
[acme@jouet linux]$
I just rebuilt the tools completely from scratch, deleting the build
dir, to see if this was some build artifact, didn't help, I still get
just the time elapsed.
Now with -v:
[acme@jouet linux]$ perf stat -v -M GFLOPs --metric-only sleep 1
Using CPUID GenuineIntel-6-3D
metric expr ( 1*( fp_arith_inst_retired.scalar_single + fp_arith_inst_retired.scalar_double ) + 2* fp_arith_inst_retired.128b_packed_double + 4*( fp_arith_inst_retired.128b_packed_single + fp_arith_inst_retired.256b_packed_double ) + 8* fp_arith_inst_retired.256b_packed_single ) / 1000000000 / duration_time for GFLOPs
found event fp_arith_inst_retired.scalar_single
found event fp_arith_inst_retired.scalar_double
found event fp_arith_inst_retired.128b_packed_double
found event fp_arith_inst_retired.128b_packed_single
found event fp_arith_inst_retired.256b_packed_double
found event fp_arith_inst_retired.256b_packed_single
found event duration_time
adding {fp_arith_inst_retired.scalar_single,fp_arith_inst_retired.scalar_double,fp_arith_inst_retired.128b_packed_double,fp_arith_inst_retired.128b_packed_single,fp_arith_inst_retired.256b_packed_double,fp_arith_inst_retired.256b_packed_single,duration_time}:W
fp_arith_inst_retired.scalar_single -> cpu/umask=0x2,period=2000003,event=0xc7/
fp_arith_inst_retired.scalar_double -> cpu/umask=0x1,period=2000003,event=0xc7/
fp_arith_inst_retired.128b_packed_double -> cpu/umask=0x4,period=2000003,event=0xc7/
fp_arith_inst_retired.128b_packed_single -> cpu/umask=0x8,period=2000003,event=0xc7/
fp_arith_inst_retired.256b_packed_double -> cpu/umask=0x10,period=2000003,event=0xc7/
fp_arith_inst_retired.256b_packed_single -> cpu/umask=0x20,period=2000003,event=0xc7/
Warning:
kernel.perf_event_paranoid=2, trying to fall back to excluding kernel samples
Warning:
kernel.perf_event_paranoid=2, trying to fall back to excluding kernel samples
Warning:
kernel.perf_event_paranoid=2, trying to fall back to excluding kernel samples
Warning:
kernel.perf_event_paranoid=2, trying to fall back to excluding kernel samples
Warning:
kernel.perf_event_paranoid=2, trying to fall back to excluding kernel samples
Weak group for fp_arith_inst_retired.scalar_single:u/7 failed
Warning:
kernel.perf_event_paranoid=2, trying to fall back to excluding kernel samples
Warning:
kernel.perf_event_paranoid=2, trying to fall back to excluding kernel samples
fp_arith_inst_retired.scalar_single:u: 0 2137416 524337
fp_arith_inst_retired.scalar_double:u: 0 1525387 1525387
fp_arith_inst_retired.128b_packed_double:u: 0 2429065 2429065
fp_arith_inst_retired.128b_packed_single:u: 0 2429065 2429065
fp_arith_inst_retired.256b_packed_double:u: 0 2429065 1904728
fp_arith_inst_retired.256b_packed_single:u: 0 2429065 903678
duration_time:u: 0 2394375 2394375
Performance counter stats for 'sleep 1':
fp_arith_inst_retired.scalar_single not found
GFLOPs
fp_arith_inst_retired.scalar_single not found
1.005367355 seconds time elapsed
[acme@jouet linux]$
We need to improve these messages...
As root for that Summary more:
[acme@jouet linux]$ sudo ~acme/bin/perf stat -M Summary --metric-only -a sleep 1
Performance counter stats for 'system wide':
Instructions CPI CLKS CPU_Utilization GFLOPs SMT_2T_Utilization Kernel_Utilization
134707420.0 0.0 698757419.0 0.1 0.0 0.3 0.4
1.003365814 seconds time elapsed
[acme@jouet linux]$
- Arnaldo
[toc] | [prev] | [next] | [standalone]
| From | Andi Kleen <ak@linux.intel.com> |
|---|---|
| Date | 2017-09-05 20:30 +0200 |
| Subject | Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat |
| Message-ID | <umqLL-6oD-3@gated-at.bofh.it> |
| In reply to | #1726909 |
On Tue, Sep 05, 2017 at 03:09:19PM -0300, Arnaldo Carvalho de Melo wrote:
> Em Mon, Sep 04, 2017 at 10:37:25AM -0700, Andi Kleen escreveu:
> > On Mon, Sep 04, 2017 at 02:11:28PM -0300, Arnaldo Carvalho de Melo wrote:
> > > Em Thu, Aug 31, 2017 at 12:40:31PM -0700, Andi Kleen escreveu:
> > > > The actual JSON metrics are in a separate pull request.
> > >
> > > Yeah, I noticed when trying to test it :-\
> > >
> > > Was this pull req submitted?
> >
> > Not yet. I was waiting to finish review.
> >
> > It makes perf crash until you apply the metricgroup patchkit first.
> >
> > But you can get it here
> >
> > git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-misc.git perf/intel-json-metrics-2
>
> Ok, so I tried installing the Broadwell ones (one of my test machines)
> and got into:
I would suggest to apply all the patches from the patchkit before you test.
Some of the later ones add/fix stuff needed by some of the metrics.
>
> [root@jouet ~]# perf stat -M Pipeline flops
> bpf: builtin compilation failed: -95, try external compiler
> ERROR: problems with path cpu/uops_executed.c: No such file or directory
> Cannot set up events {uops_retired.retire_slots,inst_retired.any}:W,{inst_retired.any,cycles}:W,{uops_executed.thread,cpu/uops_executed.core,cmask=1/,uops_executed.cycles_ge_1_uop_exec}:W
>
> Usage: perf stat [<options>] [<command>]
>
> -M, --metrics <metric/metric group list>
> monitor specified metrics or metric groups (separated by ,)
> [root@jouet ~]# perf stat -M Pipeline flops
>
>
> Wasn't this already fixed?
Yes that was supposed to be fixed with
commit 77d0871c76bad1093a3d86870fe76dd1ad0ca397
Author: Andi Kleen <ak@linux.intel.com>
Date: Fri Aug 11 16:26:19 2017 -0700
perf bpf: Tighten detection of BPF events
perf stat -e cpu/uops_executed.core,cmask=1/
I'll check.
-Andi
[toc] | [prev] | [next] | [standalone]
| From | Arnaldo Carvalho de Melo <acme@kernel.org> |
|---|---|
| Date | 2017-09-05 21:00 +0200 |
| Subject | Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat |
| Message-ID | <umreO-6C4-27@gated-at.bofh.it> |
| In reply to | #1726919 |
Em Tue, Sep 05, 2017 at 11:19:52AM -0700, Andi Kleen escreveu:
> On Tue, Sep 05, 2017 at 03:09:19PM -0300, Arnaldo Carvalho de Melo wrote:
> > Em Mon, Sep 04, 2017 at 10:37:25AM -0700, Andi Kleen escreveu:
> > > But you can get it here
> > > git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-misc.git perf/intel-json-metrics-2
> > Ok, so I tried installing the Broadwell ones (one of my test machines)
> > and got into:
> I would suggest to apply all the patches from the patchkit before you test.
> Some of the later ones add/fix stuff needed by some of the metrics.
>
> >
> > [root@jouet ~]# perf stat -M Pipeline flops
> > bpf: builtin compilation failed: -95, try external compiler
> > ERROR: problems with path cpu/uops_executed.c: No such file or directory
> > Cannot set up events {uops_retired.retire_slots,inst_retired.any}:W,{inst_retired.any,cycles}:W,{uops_executed.thread,cpu/uops_executed.core,cmask=1/,uops_executed.cycles_ge_1_uop_exec}:W
> >
> > Usage: perf stat [<options>] [<command>]
> >
> > -M, --metrics <metric/metric group list>
> > monitor specified metrics or metric groups (separated by ,)
> > [root@jouet ~]# perf stat -M Pipeline flops
> >
> >
> > Wasn't this already fixed?
>
> Yes that was supposed to be fixed with
> perf bpf: Tighten detection of BPF events
>
> perf stat -e cpu/uops_executed.core,cmask=1/
> I'll check.
Ok, I couldn't reproduce it anymore, after you check this, please let me
know if I should pull that perf/intel-json-metrics-2 branch so that
someone wanting to test this can have it all in one place, ok?
- Arnaldo
[toc] | [prev] | [next] | [standalone]
| From | Andi Kleen <andi@firstfloor.org> |
|---|---|
| Date | 2017-09-05 22:00 +0200 |
| Subject | Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat |
| Message-ID | <umsaR-7dY-3@gated-at.bofh.it> |
| In reply to | #1726934 |
> > I'll check. > > Ok, I couldn't reproduce it anymore, after you check this, please let me > know if I should pull that perf/intel-json-metrics-2 branch so that > someone wanting to test this can have it all in one place, ok? Sure please pull. The only missing thing are metrics for Skylake Server, but I can submit those later. -Andi
[toc] | [prev] | [next] | [standalone]
| From | Arnaldo Carvalho de Melo <acme@kernel.org> |
|---|---|
| Date | 2017-09-05 22:10 +0200 |
| Subject | Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat |
| Message-ID | <umskx-7wQ-17@gated-at.bofh.it> |
| In reply to | #1726960 |
Em Tue, Sep 05, 2017 at 12:52:35PM -0700, Andi Kleen escreveu: > > > I'll check. > > > > Ok, I couldn't reproduce it anymore, after you check this, please let me > > know if I should pull that perf/intel-json-metrics-2 branch so that > > someone wanting to test this can have it all in one place, ok? > > Sure please pull. > > The only missing thing are metrics for Skylake Server, but I can > submit those later. Ok, so I looked at the commit logs and found them rather dull, how is that option to fake a CPU so that the tool thinks it is running on some specific machine (broadwell, skylake, etc) so that I can augment those with the output of 'perf list metricgroup' for each of them? - Arnaldo
[toc] | [prev] | [next] | [standalone]
| From | Andi Kleen <ak@linux.intel.com> |
|---|---|
| Date | 2017-09-05 22:40 +0200 |
| Subject | Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat |
| Message-ID | <umsNA-7IG-27@gated-at.bofh.it> |
| In reply to | #1726963 |
On Tue, Sep 05, 2017 at 05:07:09PM -0300, Arnaldo Carvalho de Melo wrote: > Em Tue, Sep 05, 2017 at 12:52:35PM -0700, Andi Kleen escreveu: > > > > I'll check. > > > > > > Ok, I couldn't reproduce it anymore, after you check this, please let me > > > know if I should pull that perf/intel-json-metrics-2 branch so that > > > someone wanting to test this can have it all in one place, ok? > > > > Sure please pull. > > > > The only missing thing are metrics for Skylake Server, but I can > > submit those later. > > Ok, so I looked at the commit logs and found them rather dull, how is > that option to fake a CPU so that the tool thinks it is running on some > specific machine (broadwell, skylake, etc) so that I can augment those > with the output of 'perf list metricgroup' for each of them? PERF_CPUID=GenuineIntel-... See the mapfile.csv for valid codes But it's quite a few. -Andi
[toc] | [prev] | [next] | [standalone]
| From | Arnaldo Carvalho de Melo <acme@kernel.org> |
|---|---|
| Date | 2017-09-08 20:20 +0200 |
| Subject | Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat |
| Message-ID | <unw2K-283-21@gated-at.bofh.it> |
| In reply to | #1726976 |
Em Tue, Sep 05, 2017 at 01:37:08PM -0700, Andi Kleen escreveu:
> On Tue, Sep 05, 2017 at 05:07:09PM -0300, Arnaldo Carvalho de Melo wrote:
> > Em Tue, Sep 05, 2017 at 12:52:35PM -0700, Andi Kleen escreveu:
> > > > > I'll check.
> > > >
> > > > Ok, I couldn't reproduce it anymore, after you check this, please let me
> > > > know if I should pull that perf/intel-json-metrics-2 branch so that
> > > > someone wanting to test this can have it all in one place, ok?
> > >
> > > Sure please pull.
> > >
> > > The only missing thing are metrics for Skylake Server, but I can
> > > submit those later.
> >
> > Ok, so I looked at the commit logs and found them rather dull, how is
> > that option to fake a CPU so that the tool thinks it is running on some
> > specific machine (broadwell, skylake, etc) so that I can augment those
> > with the output of 'perf list metricgroup' for each of them?
>
> PERF_CPUID=GenuineIntel-...
>
> See the mapfile.csv for valid codes
>
> But it's quite a few.
yeah, I'm testing on the ones I have access and on a Skylake machine I'm having
trouble, see below.
That error message is not that helpful, what could be at play here? The kernel
running there is a recent RT variant for the RHEL kernel, with lots of
backports in the perf codebase, so I wasn't expecting that to be an issue,
please take a look.
I'll try later with a more recent kernel, maybe there are kernel patches
supporting Skylake that aren't in this kernel :-\
[root@seventh ~]# uname -a
Linux seventh 3.10.0-703.rt56.630.el7.x86_64 #1 SMP PREEMPT RT Sat Aug 19 22:32:17 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux
[root@seventh ~]# grep "model name" /proc/cpuinfo | head -1
model name : Intel(R) Core(TM) i5-7500 CPU @ 3.40GHz
[root@seventh ~]# perf list metricgroup
List of pre-defined events (to be used in -e):
Metric Groups:
DSB
FLOPS
Frontend
Frontend_Bandwidth
Memory_BW
Memory_Bound
Memory_Lat
Pipeline
Ports_Utilization
Power
SMT
Summary
TLB
TopDownL1
Unknown_Branches
[root@seventh ~]# perf stat --metric-only -M Summary -a sleep 1
Cannot set up events {inst_retired.any,cycles}:W,{cpu_clk_unhalted.thread}:W,{inst_retired.any}:W,{cpu_clk_unhalted.ref_tsc,msr/tsc/}:W,{fp_arith_inst_retired.scalar_single,fp_arith_inst_retired.scalar_double,fp_arith_inst_retired.128b_packed_double,fp_arith_inst_retired.128b_packed_single,fp_arith_inst_retired.256b_packed_double,fp_arith_inst_retired.256b_packed_single,duration_time}:W,{cpu_clk_thread_unhalted.one_thread_active,cpu_clk_thread_unhalted.ref_xclk_any}:W,{cpu_clk_unhalted.ref_tsc:u,cpu_clk_unhalted.ref_tsc}:W
Usage: perf stat [<options>] [<command>]
-M, --metrics <metric/metric group list>
monitor specified metrics or metric groups (separated by ,)
[root@seventh ~]#
[root@seventh ~]# perf stat -vv --metric-only -M Summary -a sleep 1
Using CPUID GenuineIntel-6-9E
metric expr 1 / inst_retired.any / cycles for CPI
found event inst_retired.any
found event cycles
metric expr cpu_clk_unhalted.thread for CLKS
found event cpu_clk_unhalted.thread
metric expr inst_retired.any for Instructions
found event inst_retired.any
metric expr cpu_clk_unhalted.ref_tsc / msr@tsc@ for CPU_Utilization
found event cpu_clk_unhalted.ref_tsc
found event msr/tsc/
metric expr ( 1*( fp_arith_inst_retired.scalar_single + fp_arith_inst_retired.scalar_double ) + 2* fp_arith_inst_retired.128b_packed_double + 4*( fp_arith_inst_retired.128b_packed_single + fp_arith_inst_retired.256b_packed_double ) + 8* fp_arith_inst_retired.256b_packed_single ) / 1000000000 / duration_time for GFLOPs
found event fp_arith_inst_retired.scalar_single
found event fp_arith_inst_retired.scalar_double
found event fp_arith_inst_retired.128b_packed_double
found event fp_arith_inst_retired.128b_packed_single
found event fp_arith_inst_retired.256b_packed_double
found event fp_arith_inst_retired.256b_packed_single
found event duration_time
metric expr 1 - cpu_clk_thread_unhalted.one_thread_active / ( cpu_clk_thread_unhalted.ref_xclk_any / 2 ) if #smt_on else 0 for SMT_2T_Utilization
found event cpu_clk_thread_unhalted.one_thread_active
found event cpu_clk_thread_unhalted.ref_xclk_any
metric expr cpu_clk_unhalted.ref_tsc:u / cpu_clk_unhalted.ref_tsc for Kernel_Utilization
found event cpu_clk_unhalted.ref_tsc:u
found event cpu_clk_unhalted.ref_tsc
adding {inst_retired.any,cycles}:W,{cpu_clk_unhalted.thread}:W,{inst_retired.any}:W,{cpu_clk_unhalted.ref_tsc,msr/tsc/}:W,{fp_arith_inst_retired.scalar_single,fp_arith_inst_retired.scalar_double,fp_arith_inst_retired.128b_packed_double,fp_arith_inst_retired.128b_packed_single,fp_arith_inst_retired.256b_packed_double,fp_arith_inst_retired.256b_packed_single,duration_time}:W,{cpu_clk_thread_unhalted.one_thread_active,cpu_clk_thread_unhalted.ref_xclk_any}:W,{cpu_clk_unhalted.ref_tsc:u,cpu_clk_unhalted.ref_tsc}:W
intel_pt default config: tsc,mtc,mtc_period=3,psb_period=3
inst_retired.any -> cpu/event=0xc0/
cpu_clk_unhalted.thread -> cpu/event=0x3c/
inst_retired.any -> cpu/event=0xc0/
cpu_clk_unhalted.ref_tsc -> cpu/umask=0x3,period=2000003,event=0/
Cannot set up events {inst_retired.any,cycles}:W,{cpu_clk_unhalted.thread}:W,{inst_retired.any}:W,{cpu_clk_unhalted.ref_tsc,msr/tsc/}:W,{fp_arith_inst_retired.scalar_single,fp_arith_inst_retired.scalar_double,fp_arith_inst_retired.128b_packed_double,fp_arith_inst_retired.128b_packed_single,fp_arith_inst_retired.256b_packed_double,fp_arith_inst_retired.256b_packed_single,duration_time}:W,{cpu_clk_thread_unhalted.one_thread_active,cpu_clk_thread_unhalted.ref_xclk_any}:W,{cpu_clk_unhalted.ref_tsc:u,cpu_clk_unhalted.ref_tsc}:W
Usage: perf stat [<options>] [<command>]
-M, --metrics <metric/metric group list>
monitor specified metrics or metric groups (separated by ,)
[root@seventh ~]#
Humm, it doesn't even get to try sys_perf_event_open() it seems to be bailing
out while reading some sysfs event_source files, a better error output seems in
demand:
[root@seventh ~]# perf trace -e perf_* perf stat --metric-only -M Summary -a sleep 1
Cannot set up events {inst_retired.any,cycles}:W,{cpu_clk_unhalted.thread}:W,{inst_retired.any}:W,{cpu_clk_unhalted.ref_tsc,msr/tsc/}:W,{fp_arith_inst_retired.scalar_single,fp_arith_inst_retired.scalar_double,fp_arith_inst_retired.128b_packed_double,fp_arith_inst_retired.128b_packed_single,fp_arith_inst_retired.256b_packed_double,fp_arith_inst_retired.256b_packed_single,duration_time}:W,{cpu_clk_thread_unhalted.one_thread_active,cpu_clk_thread_unhalted.ref_xclk_any}:W,{cpu_clk_unhalted.ref_tsc:u,cpu_clk_unhalted.ref_tsc}:W
Usage: perf stat [<options>] [<command>]
-M, --metrics <metric/metric group list>
monitor specified metrics or metric groups (separated by ,)
[root@seventh ~]#
[root@seventh linux]# strace perf stat --metric-only -M Summary -a sleep 1 |& tail -30
getdents(3, /* 0 entries */, 32768) = 0
close(3) = 0
openat(AT_FDCWD, "/sys/bus/event_source/devices/", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3
getdents(3, /* 8 entries */, 32768) = 232
getdents(3, /* 0 entries */, 32768) = 0
close(3) = 0
stat("/sys/bus/event_source/devices/msr/format", 0x7ffd36f79480) = -1 ENOENT (No such file or directory)
stat("/sys/bus/event_source/devices/msr/type", 0x7ffd36f79470) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/sys/bus/event_source/devices/", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3
getdents(3, /* 8 entries */, 32768) = 232
getdents(3, /* 0 entries */, 32768) = 0
close(3) = 0
write(2, "Cannot set up events {inst_retir"..., 529Cannot set up events {inst_retired.any,cycles}:W,{cpu_clk_unhalted.thread}:W,{inst_retired.any}:W,{cpu_clk_unhalted.ref_tsc,msr/tsc/}:W,{fp_arith_inst_retired.scalar_single,fp_arith_inst_retired.scalar_double,fp_arith_inst_retired.128b_packed_double,fp_arith_inst_retired.128b_packed_single,fp_arith_inst_retired.256b_packed_double,fp_arith_inst_retired.256b_packed_single,duration_time}:W,{cpu_clk_thread_unhalted.one_thread_active,cpu_clk_thread_unhalted.ref_xclk_any}:W,{cpu_clk_unhalted.ref_tsc:u,cpu_clk_unhalted.ref_tsc}:W
) = 529
write(2, "\n Usage: perf stat [<options>] ["..., 43
Usage: perf stat [<options>] [<command>]
) = 43
write(2, "\n", 1
) = 1
write(2, " ", 4 ) = 4
write(2, "-M", 2-M) = 2
write(2, ", ", 2, ) = 2
write(2, "--metrics", 9--metrics) = 9
write(2, " <metric/metric group list>", 27 <metric/metric group list>) = 27
write(2, "\n", 1
) = 1
write(2, " monito"..., 86 monitor specified metrics or metric groups (separated by ,)
) = 86
exit_group(129) = ?
+++ exited with 129 +++
[root@seventh linux]#
[toc] | [prev] | [next] | [standalone]
| From | Andi Kleen <ak@linux.intel.com> |
|---|---|
| Date | 2017-09-08 21:10 +0200 |
| Subject | Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat |
| Message-ID | <unwP7-2If-5@gated-at.bofh.it> |
| In reply to | #1729172 |
> > PERF_CPUID=GenuineIntel-... > > > > See the mapfile.csv for valid codes > > > > But it's quite a few. > > yeah, I'm testing on the ones I have access and on a Skylake machine I'm having > trouble, see below. > [root@seventh ~]# perf stat -vv --metric-only -M Summary -a sleep 1 The problem is that your kernel is missing the msr// PMU. It should work with a newer kernel. I can look at better error messages. -Andi
[toc] | [prev] | [next] | [standalone]
| From | Andi Kleen <andi@firstfloor.org> |
|---|---|
| Date | 2017-08-31 21:50 +0200 |
| Subject | [PATCH v3 04/11] perf, tools: Print generic metric header even for failed expressions |
| Message-ID | <ukDDs-1ub-33@gated-at.bofh.it> |
| In reply to | #1724527 |
From: Andi Kleen <ak@linux.intel.com> Print the generic metric header even when the expression evaluation failed. Otherwise an expression that fails on the first collections due to division by zero may suddenly reappear later without an header. Signed-off-by: Andi Kleen <ak@linux.intel.com> --- tools/perf/util/stat-shadow.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tools/perf/util/stat-shadow.c b/tools/perf/util/stat-shadow.c index 96aa6cbf24d6..8c7ab29169b9 100644 --- a/tools/perf/util/stat-shadow.c +++ b/tools/perf/util/stat-shadow.c @@ -662,7 +662,9 @@ static void generic_metric(const char *metric_expr, out->force_header ? name : "", ratio); else - print_metric(ctxp, NULL, NULL, "", 0); + print_metric(ctxp, NULL, NULL, + out->force_header ? + (metric_name ? metric_name : name) : "", 0); } else print_metric(ctxp, NULL, NULL, "", 0); } -- 2.9.5
[toc] | [prev] | [next] | [standalone]
| From | Andi Kleen <andi@firstfloor.org> |
|---|---|
| Date | 2017-08-31 21:50 +0200 |
| Subject | [PATCH v3 10/11] perf, tools, stat: Hide internal duration_time counter |
| Message-ID | <ukDDs-1ub-31@gated-at.bofh.it> |
| In reply to | #1724527 |
From: Andi Kleen <ak@linux.intel.com>
Some perf stat metrics use an internal "duration_time" metric. It is not
correctly printed however. So hide it during output to avoid confusing users
with 0 counts.
Signed-off-by: Andi Kleen <ak@linux.intel.com>
---
tools/perf/builtin-stat.c | 27 ++++++++++++++++++++++++---
1 file changed, 24 insertions(+), 3 deletions(-)
diff --git a/tools/perf/builtin-stat.c b/tools/perf/builtin-stat.c
index 045efd7d7785..bf6ae0144ecf 100644
--- a/tools/perf/builtin-stat.c
+++ b/tools/perf/builtin-stat.c
@@ -195,6 +195,11 @@ static struct perf_stat_config stat_config = {
.scale = true,
};
+static bool is_duration_time(struct perf_evsel *evsel)
+{
+ return !strcmp(evsel->name, "duration_time");
+}
+
static inline void diff_timespec(struct timespec *r, struct timespec *a,
struct timespec *b)
{
@@ -1363,6 +1368,9 @@ static void print_aggr(char *prefix)
ad.id = id = aggr_map->map[s];
first = true;
evlist__for_each_entry(evsel_list, counter) {
+ if (is_duration_time(counter))
+ continue;
+
ad.val = ad.ena = ad.run = 0;
ad.nr = 0;
if (!collect_data(counter, aggr_cb, &ad))
@@ -1506,6 +1514,8 @@ static void print_no_aggr_metric(char *prefix)
if (prefix)
fputs(prefix, stat_config.output);
evlist__for_each_entry(evsel_list, counter) {
+ if (is_duration_time(counter))
+ continue;
if (first) {
aggr_printout(counter, cpu, 0);
first = false;
@@ -1560,6 +1570,8 @@ static void print_metric_headers(const char *prefix, bool no_indent)
/* Print metrics headers only */
evlist__for_each_entry(evsel_list, counter) {
+ if (is_duration_time(counter))
+ continue;
os.evsel = counter;
out.ctx = &os;
out.print_metric = print_metric_header;
@@ -1707,12 +1719,18 @@ static void print_counters(struct timespec *ts, int argc, const char **argv)
print_aggr(prefix);
break;
case AGGR_THREAD:
- evlist__for_each_entry(evsel_list, counter)
+ evlist__for_each_entry(evsel_list, counter) {
+ if (is_duration_time(counter))
+ continue;
print_aggr_thread(counter, prefix);
+ }
break;
case AGGR_GLOBAL:
- evlist__for_each_entry(evsel_list, counter)
+ evlist__for_each_entry(evsel_list, counter) {
+ if (is_duration_time(counter))
+ continue;
print_counter_aggr(counter, prefix);
+ }
if (metric_only)
fputc('\n', stat_config.output);
break;
@@ -1720,8 +1738,11 @@ static void print_counters(struct timespec *ts, int argc, const char **argv)
if (metric_only)
print_no_aggr_metric(prefix);
else {
- evlist__for_each_entry(evsel_list, counter)
+ evlist__for_each_entry(evsel_list, counter) {
+ if (is_duration_time(counter))
+ continue;
print_counter(counter, prefix);
+ }
}
break;
case AGGR_UNSET:
--
2.9.5
[toc] | [prev] | [next] | [standalone]
| From | Andi Kleen <andi@firstfloor.org> |
|---|---|
| Date | 2017-08-31 21:50 +0200 |
| Subject | [PATCH v3 09/11] perf, tools, stat: Support duration_time for metrics |
| Message-ID | <ukDDs-1ub-35@gated-at.bofh.it> |
| In reply to | #1724527 |
From: Andi Kleen <ak@linux.intel.com>
Some of the metrics formulas (like GFLOPs) need to know how long
the measurement period is. Support an internal event called duration_time, which
reports time in second. It maps to the dummy event, but is
special cased for statistics to report the walltime duration.
So far it is not printed, but only used internally for metrics.
Signed-off-by: Andi Kleen <ak@linux.intel.com>
---
tools/perf/util/parse-events.l | 1 +
tools/perf/util/stat-shadow.c | 17 +++++++++++++----
2 files changed, 14 insertions(+), 4 deletions(-)
diff --git a/tools/perf/util/parse-events.l b/tools/perf/util/parse-events.l
index fdb5bb52f01f..ea2426daf7e8 100644
--- a/tools/perf/util/parse-events.l
+++ b/tools/perf/util/parse-events.l
@@ -288,6 +288,7 @@ cpu-migrations|migrations { return sym(yyscanner, PERF_TYPE_SOFTWARE, PERF_COU
alignment-faults { return sym(yyscanner, PERF_TYPE_SOFTWARE, PERF_COUNT_SW_ALIGNMENT_FAULTS); }
emulation-faults { return sym(yyscanner, PERF_TYPE_SOFTWARE, PERF_COUNT_SW_EMULATION_FAULTS); }
dummy { return sym(yyscanner, PERF_TYPE_SOFTWARE, PERF_COUNT_SW_DUMMY); }
+duration_time { return sym(yyscanner, PERF_TYPE_SOFTWARE, PERF_COUNT_SW_DUMMY); }
bpf-output { return sym(yyscanner, PERF_TYPE_SOFTWARE, PERF_COUNT_SW_BPF_OUTPUT); }
/*
diff --git a/tools/perf/util/stat-shadow.c b/tools/perf/util/stat-shadow.c
index 664f49a9b012..a2c12d1ef32a 100644
--- a/tools/perf/util/stat-shadow.c
+++ b/tools/perf/util/stat-shadow.c
@@ -641,11 +641,20 @@ static void generic_metric(const char *metric_expr,
expr__add_id(&pctx, name, avg);
for (i = 0; metric_events[i]; i++) {
struct saved_value *v;
+ struct stats *stats;
+ double scale;
- v = saved_value_lookup(metric_events[i], cpu, false);
- if (!v)
- break;
- expr__add_id(&pctx, metric_events[i]->name, avg_stats(&v->stats));
+ if (!strcmp(metric_events[i]->name, "duration_time")) {
+ stats = &walltime_nsecs_stats;
+ scale = 1e-9;
+ } else {
+ v = saved_value_lookup(metric_events[i], cpu, false);
+ if (!v)
+ break;
+ stats = &v->stats;
+ scale = 1.0;
+ }
+ expr__add_id(&pctx, metric_events[i]->name, avg_stats(stats)*scale);
}
if (!metric_events[i]) {
const char *p = metric_expr;
--
2.9.5
[toc] | [prev] | [next] | [standalone]
| From | Andi Kleen <andi@firstfloor.org> |
|---|---|
| Date | 2017-08-31 21:50 +0200 |
| Subject | [PATCH v3 01/11] perf, tools: Support weak groups |
| Message-ID | <ukDDt-1ub-37@gated-at.bofh.it> |
| In reply to | #1724527 |
From: Andi Kleen <ak@linux.intel.com>
Setting up groups can be complicated due to the
complicated scheduling restrictions of different PMUs.
User tools usually don't understand all these restrictions.
Still in many cases it is useful to set up groups and
they work most of the time. However if the group
is set up wrong some members will not reported any values
because they never get scheduled.
Add a concept of a 'weak group': try to set up a group,
but if it's not schedulable fallback to not using
a group. That gives us the best of both worlds:
groups if they work, but still a usable fallback if they don't.
In theory it would be possible to have more complex fallback
strategies (e.g. try to split the group in half), but
the simple fallback of not using a group seems to work for now.
So far the weak group is only implemented for perf stat,
not for record.
Here's an unschedulable group (on IvyBridge with SMT on)
% perf stat -e '{branches,branch-misses,l1d.replacement,l2_lines_in.all,l2_rqsts.all_code_rd}' -a sleep 1
73,806,067 branches
4,848,144 branch-misses # 6.57% of all branches
14,754,458 l1d.replacement
24,905,558 l2_lines_in.all
<not supported> l2_rqsts.all_code_rd <------- will never report anything
With the weak group:
% perf stat -e '{branches,branch-misses,l1d.replacement,l2_lines_in.all,l2_rqsts.all_code_rd}:W' -a sleep 1
125,366,055 branches (80.02%)
9,208,402 branch-misses # 7.35% of all branches (80.01%)
24,560,249 l1d.replacement (80.00%)
43,174,971 l2_lines_in.all (80.05%)
31,891,457 l2_rqsts.all_code_rd (79.92%)
The extra event scheduled with some extra multiplexing
v2: Move fallback code to separate function.
Add comment on for_each_group_member
Adjust to new perf_evsel__close interface
v3:
Fix debug print out.
Signed-off-by: Andi Kleen <ak@linux.intel.com>
---
tools/perf/Documentation/perf-list.txt | 1 +
tools/perf/builtin-stat.c | 35 ++++++++++++++++++++++++++++++++++
tools/perf/util/evsel.h | 1 +
tools/perf/util/parse-events.c | 8 +++++++-
tools/perf/util/parse-events.l | 2 +-
5 files changed, 45 insertions(+), 2 deletions(-)
diff --git a/tools/perf/Documentation/perf-list.txt b/tools/perf/Documentation/perf-list.txt
index f709de54707b..d432965d728d 100644
--- a/tools/perf/Documentation/perf-list.txt
+++ b/tools/perf/Documentation/perf-list.txt
@@ -47,6 +47,7 @@ counted. The following modifiers exist:
P - use maximum detected precise level
S - read sample value (PERF_SAMPLE_READ)
D - pin the event to the PMU
+ W - group is weak and will fallback to non-group if not schedulable
The 'p' modifier can be used for specifying how precise the instruction
address should be. The 'p' modifier can be specified multiple times:
diff --git a/tools/perf/builtin-stat.c b/tools/perf/builtin-stat.c
index 866da7aa54bf..501c1e0272fe 100644
--- a/tools/perf/builtin-stat.c
+++ b/tools/perf/builtin-stat.c
@@ -582,6 +582,32 @@ static bool perf_evsel__should_store_id(struct perf_evsel *counter)
return STAT_RECORD || counter->attr.read_format & PERF_FORMAT_ID;
}
+static struct perf_evsel *reset_weak_group(struct perf_evsel *counter)
+{
+ struct perf_evsel *c2, *leader;
+ bool is_open = true;
+
+ leader = counter->leader;
+ pr_debug("Weak group for %s/%d failed\n",
+ leader->name, leader->nr_members);
+
+ /*
+ * for_each_group_member doesn't work here because it doesn't
+ * include the first entry.
+ */
+ evlist__for_each_entry(evsel_list, c2) {
+ if (c2 == counter)
+ is_open = false;
+ if (c2->leader == leader) {
+ if (is_open)
+ perf_evsel__close(c2);
+ c2->leader = c2;
+ c2->nr_members = 0;
+ }
+ }
+ return leader;
+}
+
static int __run_perf_stat(int argc, const char **argv)
{
int interval = stat_config.interval;
@@ -618,6 +644,15 @@ static int __run_perf_stat(int argc, const char **argv)
evlist__for_each_entry(evsel_list, counter) {
try_again:
if (create_perf_stat_counter(counter) < 0) {
+
+ /* Weak group failed. Reset the group. */
+ if (errno == EINVAL &&
+ counter->leader != counter &&
+ counter->weak_group) {
+ counter = reset_weak_group(counter);
+ goto try_again;
+ }
+
/*
* PPC returns ENXIO for HW counters until 2.6.37
* (behavior changed with commit b0a873e).
diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h
index 351d3b2d8887..f538c3530227 100644
--- a/tools/perf/util/evsel.h
+++ b/tools/perf/util/evsel.h
@@ -136,6 +136,7 @@ struct perf_evsel {
const char * metric_name;
struct perf_evsel **metric_events;
bool collect_stat;
+ bool weak_group;
};
union u64_swap {
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index f44aeba51d1f..cd80c4eac569 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -1358,6 +1358,7 @@ struct event_modifier {
int exclude_GH;
int sample_read;
int pinned;
+ int weak;
};
static int get_event_modifier(struct event_modifier *mod, char *str,
@@ -1376,6 +1377,7 @@ static int get_event_modifier(struct event_modifier *mod, char *str,
int exclude = eu | ek | eh;
int exclude_GH = evsel ? evsel->exclude_GH : 0;
+ int weak = 0;
memset(mod, 0, sizeof(*mod));
@@ -1413,6 +1415,8 @@ static int get_event_modifier(struct event_modifier *mod, char *str,
sample_read = 1;
} else if (*str == 'D') {
pinned = 1;
+ } else if (*str == 'W') {
+ weak = 1;
} else
break;
@@ -1443,6 +1447,7 @@ static int get_event_modifier(struct event_modifier *mod, char *str,
mod->exclude_GH = exclude_GH;
mod->sample_read = sample_read;
mod->pinned = pinned;
+ mod->weak = weak;
return 0;
}
@@ -1456,7 +1461,7 @@ static int check_modifier(char *str)
char *p = str;
/* The sizeof includes 0 byte as well. */
- if (strlen(str) > (sizeof("ukhGHpppPSDI") - 1))
+ if (strlen(str) > (sizeof("ukhGHpppPSDIW") - 1))
return -1;
while (*p) {
@@ -1496,6 +1501,7 @@ int parse_events__modifier_event(struct list_head *list, char *str, bool add)
evsel->exclude_GH = mod.exclude_GH;
evsel->sample_read = mod.sample_read;
evsel->precise_max = mod.precise_max;
+ evsel->weak_group = mod.weak;
if (perf_evsel__is_group_leader(evsel))
evsel->attr.pinned = mod.pinned;
diff --git a/tools/perf/util/parse-events.l b/tools/perf/util/parse-events.l
index c42edeac451f..fdb5bb52f01f 100644
--- a/tools/perf/util/parse-events.l
+++ b/tools/perf/util/parse-events.l
@@ -161,7 +161,7 @@ name [a-zA-Z_*?][a-zA-Z0-9_*?.]*
name_minus [a-zA-Z_*?][a-zA-Z0-9\-_*?.:]*
drv_cfg_term [a-zA-Z0-9_\.]+(=[a-zA-Z0-9_*?\.:]+)?
/* If you add a modifier you need to update check_modifier() */
-modifier_event [ukhpPGHSDI]+
+modifier_event [ukhpPGHSDIW]+
modifier_bp [rwx]{1,3}
%%
--
2.9.5
[toc] | [prev] | [next] | [standalone]
Page 1 of 2 [1] 2 Next page →
Back to top | Article view | linux.kernel
csiph-web