Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1724527 > unrolled thread
| Started by | Andi Kleen <andi@firstfloor.org> |
|---|---|
| First post | 2017-08-31 21:50 +0200 |
| Last post | 2017-09-01 20:00 +0200 |
| Articles | 8 on this page of 28 — 4 participants |
Back to article view | Back to linux.kernel
Support standalone metrics and metric groups for perf Andi Kleen <andi@firstfloor.org> - 2017-08-31 21:50 +0200
[PATCH v3 07/11] perf, tools, list: Add metric groups to perf list Andi Kleen <andi@firstfloor.org> - 2017-08-31 21:50 +0200
[PATCH v3 11/11] perf, tools, stat: Update walltime_nsecs_stats in interval mode Andi Kleen <andi@firstfloor.org> - 2017-08-31 21:50 +0200
[PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat Andi Kleen <andi@firstfloor.org> - 2017-08-31 21:50 +0200
Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat Arnaldo Carvalho de Melo <acme@kernel.org> - 2017-09-04 19:20 +0200
Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat Andi Kleen <andi@firstfloor.org> - 2017-09-04 19:40 +0200
Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat Arnaldo Carvalho de Melo <acme@kernel.org> - 2017-09-05 20:10 +0200
Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat Arnaldo Carvalho de Melo <acme@kernel.org> - 2017-09-05 20:20 +0200
Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat Arnaldo Carvalho de Melo <acme@kernel.org> - 2017-09-05 20:40 +0200
Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat Andi Kleen <ak@linux.intel.com> - 2017-09-05 20:30 +0200
Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat Arnaldo Carvalho de Melo <acme@kernel.org> - 2017-09-05 21:00 +0200
Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat Andi Kleen <andi@firstfloor.org> - 2017-09-05 22:00 +0200
Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat Arnaldo Carvalho de Melo <acme@kernel.org> - 2017-09-05 22:10 +0200
Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat Andi Kleen <ak@linux.intel.com> - 2017-09-05 22:40 +0200
Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat Arnaldo Carvalho de Melo <acme@kernel.org> - 2017-09-08 20:20 +0200
Re: [PATCH v3 06/11] perf, tools, stat: Support JSON metrics in perf stat Andi Kleen <ak@linux.intel.com> - 2017-09-08 21:10 +0200
[PATCH v3 04/11] perf, tools: Print generic metric header even for failed expressions Andi Kleen <andi@firstfloor.org> - 2017-08-31 21:50 +0200
[PATCH v3 10/11] perf, tools, stat: Hide internal duration_time counter Andi Kleen <andi@firstfloor.org> - 2017-08-31 21:50 +0200
[PATCH v3 09/11] perf, tools, stat: Support duration_time for metrics Andi Kleen <andi@firstfloor.org> - 2017-08-31 21:50 +0200
[PATCH v3 01/11] perf, tools: Support weak groups Andi Kleen <andi@firstfloor.org> - 2017-08-31 21:50 +0200
Re: [PATCH v3 01/11] perf, tools: Support weak groups Jiri Olsa <jolsa@redhat.com> - 2017-09-01 19:00 +0200
Re: [PATCH v3 01/11] perf, tools: Support weak groups Jiri Olsa <jolsa@redhat.com> - 2017-09-01 19:10 +0200
Re: [PATCH v3 01/11] perf, tools: Support weak groups Arnaldo Carvalho de Melo <acme@kernel.org> - 2017-09-04 19:00 +0200
[PATCH v3 08/11] perf, tools, stat: Don't use ctx for saved values lookup Andi Kleen <andi@firstfloor.org> - 2017-08-31 21:50 +0200
Re: Support standalone metrics and metric groups for perf Jiri Olsa <jolsa@redhat.com> - 2017-09-01 19:30 +0200
Re: Support standalone metrics and metric groups for perf Jiri Olsa <jolsa@redhat.com> - 2017-09-01 19:40 +0200
Re: Support standalone metrics and metric groups for perf Andi Kleen <andi@firstfloor.org> - 2017-09-01 19:50 +0200
Re: Support standalone metrics and metric groups for perf Jiri Olsa <jolsa@redhat.com> - 2017-09-01 20:00 +0200
Page 2 of 2 — ← Prev page 1 [2]
| From | Jiri Olsa <jolsa@redhat.com> |
|---|---|
| Date | 2017-09-01 19:00 +0200 |
| Subject | Re: [PATCH v3 01/11] perf, tools: Support weak groups |
| Message-ID | <ukXsu-7Iv-15@gated-at.bofh.it> |
| In reply to | #1724535 |
On Thu, Aug 31, 2017 at 12:40:26PM -0700, Andi Kleen wrote:
> From: Andi Kleen <ak@linux.intel.com>
>
> Setting up groups can be complicated due to the
> complicated scheduling restrictions of different PMUs.
> User tools usually don't understand all these restrictions.
> Still in many cases it is useful to set up groups and
> they work most of the time. However if the group
> is set up wrong some members will not reported any values
> because they never get scheduled.
>
> Add a concept of a 'weak group': try to set up a group,
> but if it's not schedulable fallback to not using
> a group. That gives us the best of both worlds:
> groups if they work, but still a usable fallback if they don't.
>
> In theory it would be possible to have more complex fallback
> strategies (e.g. try to split the group in half), but
> the simple fallback of not using a group seems to work for now.
>
> So far the weak group is only implemented for perf stat,
> not for record.
>
> Here's an unschedulable group (on IvyBridge with SMT on)
>
> % perf stat -e '{branches,branch-misses,l1d.replacement,l2_lines_in.all,l2_rqsts.all_code_rd}' -a sleep 1
>
> 73,806,067 branches
> 4,848,144 branch-misses # 6.57% of all branches
> 14,754,458 l1d.replacement
> 24,905,558 l2_lines_in.all
> <not supported> l2_rqsts.all_code_rd <------- will never report anything
>
> With the weak group:
>
> % perf stat -e '{branches,branch-misses,l1d.replacement,l2_lines_in.all,l2_rqsts.all_code_rd}:W' -a sleep 1
>
> 125,366,055 branches (80.02%)
> 9,208,402 branch-misses # 7.35% of all branches (80.01%)
> 24,560,249 l1d.replacement (80.00%)
> 43,174,971 l2_lines_in.all (80.05%)
> 31,891,457 l2_rqsts.all_code_rd (79.92%)
>
> The extra event scheduled with some extra multiplexing
>
> v2: Move fallback code to separate function.
> Add comment on for_each_group_member
> Adjust to new perf_evsel__close interface
> v3:
> Fix debug print out.
> Signed-off-by: Andi Kleen <ak@linux.intel.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
thanks,
jirka
[toc] | [prev] | [next] | [standalone]
| From | Jiri Olsa <jolsa@redhat.com> |
|---|---|
| Date | 2017-09-01 19:10 +0200 |
| Subject | Re: [PATCH v3 01/11] perf, tools: Support weak groups |
| Message-ID | <ukXCa-81A-25@gated-at.bofh.it> |
| In reply to | #1725231 |
On Fri, Sep 01, 2017 at 06:57:59PM +0200, Jiri Olsa wrote:
> On Thu, Aug 31, 2017 at 12:40:26PM -0700, Andi Kleen wrote:
> > From: Andi Kleen <ak@linux.intel.com>
> >
> > Setting up groups can be complicated due to the
> > complicated scheduling restrictions of different PMUs.
> > User tools usually don't understand all these restrictions.
> > Still in many cases it is useful to set up groups and
> > they work most of the time. However if the group
> > is set up wrong some members will not reported any values
> > because they never get scheduled.
> >
> > Add a concept of a 'weak group': try to set up a group,
> > but if it's not schedulable fallback to not using
> > a group. That gives us the best of both worlds:
> > groups if they work, but still a usable fallback if they don't.
> >
> > In theory it would be possible to have more complex fallback
> > strategies (e.g. try to split the group in half), but
> > the simple fallback of not using a group seems to work for now.
> >
> > So far the weak group is only implemented for perf stat,
> > not for record.
> >
> > Here's an unschedulable group (on IvyBridge with SMT on)
> >
> > % perf stat -e '{branches,branch-misses,l1d.replacement,l2_lines_in.all,l2_rqsts.all_code_rd}' -a sleep 1
> >
> > 73,806,067 branches
> > 4,848,144 branch-misses # 6.57% of all branches
> > 14,754,458 l1d.replacement
> > 24,905,558 l2_lines_in.all
> > <not supported> l2_rqsts.all_code_rd <------- will never report anything
> >
> > With the weak group:
> >
> > % perf stat -e '{branches,branch-misses,l1d.replacement,l2_lines_in.all,l2_rqsts.all_code_rd}:W' -a sleep 1
> >
> > 125,366,055 branches (80.02%)
> > 9,208,402 branch-misses # 7.35% of all branches (80.01%)
> > 24,560,249 l1d.replacement (80.00%)
> > 43,174,971 l2_lines_in.all (80.05%)
> > 31,891,457 l2_rqsts.all_code_rd (79.92%)
> >
> > The extra event scheduled with some extra multiplexing
> >
> > v2: Move fallback code to separate function.
> > Add comment on for_each_group_member
> > Adjust to new perf_evsel__close interface
> > v3:
> > Fix debug print out.
> > Signed-off-by: Andi Kleen <ak@linux.intel.com>
>
> Acked-by: Jiri Olsa <jolsa@kernel.org>
just realized we support this in stat only and the doc
indicates it's global.. maybe the perf-list.txt line
could have the '(perf stat only)' suffix ;-)
jirka
[toc] | [prev] | [next] | [standalone]
| From | Arnaldo Carvalho de Melo <acme@kernel.org> |
|---|---|
| Date | 2017-09-04 19:00 +0200 |
| Subject | Re: [PATCH v3 01/11] perf, tools: Support weak groups |
| Message-ID | <um2T9-7VN-27@gated-at.bofh.it> |
| In reply to | #1725242 |
Em Fri, Sep 01, 2017 at 07:00:39PM +0200, Jiri Olsa escreveu: > On Fri, Sep 01, 2017 at 06:57:59PM +0200, Jiri Olsa wrote: > > On Thu, Aug 31, 2017 at 12:40:26PM -0700, Andi Kleen wrote: > > > From: Andi Kleen <ak@linux.intel.com> > > > Add a concept of a 'weak group': try to set up a group, > > > but if it's not schedulable fallback to not using > > > a group. That gives us the best of both worlds: > > > groups if they work, but still a usable fallback if they don't. > > Acked-by: Jiri Olsa <jolsa@kernel.org> > just realized we support this in stat only and the doc indicates it's > global.. maybe the perf-list.txt line could have the '(perf stat > only)' suffix ;-) I'll try to add something to that effect. - Arnaldo
[toc] | [prev] | [next] | [standalone]
| From | Andi Kleen <andi@firstfloor.org> |
|---|---|
| Date | 2017-08-31 21:50 +0200 |
| Subject | [PATCH v3 08/11] perf, tools, stat: Don't use ctx for saved values lookup |
| Message-ID | <ukDDt-1ub-41@gated-at.bofh.it> |
| In reply to | #1724527 |
From: Andi Kleen <ak@linux.intel.com>
We don't need to use ctx to look up events for saved values.
The context is already part of the evsel pointer, which is the
primary key.
Signed-off-by: Andi Kleen <ak@linux.intel.com>
---
tools/perf/util/stat-shadow.c | 16 +++++-----------
1 file changed, 5 insertions(+), 11 deletions(-)
diff --git a/tools/perf/util/stat-shadow.c b/tools/perf/util/stat-shadow.c
index 42e6c17be7ff..664f49a9b012 100644
--- a/tools/perf/util/stat-shadow.c
+++ b/tools/perf/util/stat-shadow.c
@@ -56,7 +56,6 @@ struct saved_value {
struct rb_node rb_node;
struct perf_evsel *evsel;
int cpu;
- int ctx;
struct stats stats;
};
@@ -67,8 +66,6 @@ static int saved_value_cmp(struct rb_node *rb_node, const void *entry)
rb_node);
const struct saved_value *b = entry;
- if (a->ctx != b->ctx)
- return a->ctx - b->ctx;
if (a->cpu != b->cpu)
return a->cpu - b->cpu;
if (a->evsel == b->evsel)
@@ -90,13 +87,12 @@ static struct rb_node *saved_value_new(struct rblist *rblist __maybe_unused,
}
static struct saved_value *saved_value_lookup(struct perf_evsel *evsel,
- int cpu, int ctx,
+ int cpu,
bool create)
{
struct rb_node *nd;
struct saved_value dm = {
.cpu = cpu,
- .ctx = ctx,
.evsel = evsel,
};
nd = rblist__find(&runtime_saved_values, &dm);
@@ -232,8 +228,7 @@ void perf_stat__update_shadow_stats(struct perf_evsel *counter, u64 *count,
update_stats(&runtime_aperf_stats[ctx][cpu], count[0]);
if (counter->collect_stat) {
- struct saved_value *v = saved_value_lookup(counter, cpu, ctx,
- true);
+ struct saved_value *v = saved_value_lookup(counter, cpu, true);
update_stats(&v->stats, count[0]);
}
}
@@ -634,7 +629,6 @@ static void generic_metric(const char *metric_expr,
const char *metric_name,
double avg,
int cpu,
- int ctx,
struct perf_stat_output_ctx *out)
{
print_metric_t print_metric = out->print_metric;
@@ -648,7 +642,7 @@ static void generic_metric(const char *metric_expr,
for (i = 0; metric_events[i]; i++) {
struct saved_value *v;
- v = saved_value_lookup(metric_events[i], cpu, ctx, false);
+ v = saved_value_lookup(metric_events[i], cpu, false);
if (!v)
break;
expr__add_id(&pctx, metric_events[i]->name, avg_stats(&v->stats));
@@ -866,7 +860,7 @@ void perf_stat__print_shadow_stats(struct perf_evsel *evsel,
print_metric(ctxp, NULL, NULL, name, 0);
} else if (evsel->metric_expr) {
generic_metric(evsel->metric_expr, evsel->metric_events, evsel->name,
- evsel->metric_name, avg, cpu, ctx, out);
+ evsel->metric_name, avg, cpu, out);
} else if (runtime_nsecs_stats[cpu].n != 0) {
char unit = 'M';
char unit_buf[10];
@@ -895,7 +889,7 @@ void perf_stat__print_shadow_stats(struct perf_evsel *evsel,
out->new_line(ctxp);
generic_metric(mexp->metric_expr, mexp->metric_events,
evsel->name, mexp->metric_name,
- avg, cpu, ctx, out);
+ avg, cpu, out);
}
}
if (num == 0)
--
2.9.5
[toc] | [prev] | [next] | [standalone]
| From | Jiri Olsa <jolsa@redhat.com> |
|---|---|
| Date | 2017-09-01 19:30 +0200 |
| Message-ID | <ukXVw-8aN-29@gated-at.bofh.it> |
| In reply to | #1724527 |
On Thu, Aug 31, 2017 at 12:40:25PM -0700, Andi Kleen wrote:
SNIP
>
> % perf stat -M Summary --metric-only -a sleep 1
>
> Performance counter stats for 'system wide':
>
> Instructions CLKS CPU_Utilization GFLOPs SMT_2T_Utilization Kernel_Utilization
> 317614222.0 1392930775.0 0.0 0.0 0.2 0.1
>
> 1.001497549 seconds time elapsed
>
> % perf stat -M GFLOPs flops
>
> Performance counter stats for 'flops':
>
> 3,999,541,471 fp_comp_ops_exe.sse_scalar_single # 1.2 GFLOPs (66.65%)
> 14 fp_comp_ops_exe.sse_scalar_double (66.65%)
> 0 fp_comp_ops_exe.sse_packed_double (66.67%)
> 0 fp_comp_ops_exe.sse_packed_single (66.70%)
> 0 simd_fp_256.packed_double (66.70%)
> 0 simd_fp_256.packed_single (66.67%)
looks like some events are probably crossing some
output boundaries we have:
[jolsa@krava perf]$ sudo ./perf stat -M SMT -I 1000
# time counts unit events
1.000565706 408,879,985 inst_retired.any # 0.7 CoreIPC (66.68%)
1.000565706 1,120,999,114 cpu_clk_unhalted.thread_any (66.68%)
1.000565706 701,285,312 cycles (66.68%)
1.000565706 1,148,325,740 cpu_clk_unhalted.thread_any # 574162870.0 CORE_CLKS (66.67%)
1.000565706 711,565,247 cpu_clk_unhalted.thread (66.66%)
1.000565706 24,057,590 cpu_clk_thread_unhalted.one_thread_active # 0.3 SMT_2T_Utilization (66.67%)
1.000565706 65,753,475 cpu_clk_thread_unhalted.ref_xclk_any (66.67%)
^C 1.349436822 21,198,385 inst_retired.any # 0.1 CoreIPC (66.70%)
1.349436822 112,740,282 cpu_clk_unhalted.thread_any (66.70%)
1.349436822 84,509,414 cycles (66.70%)
1.349436822 108,181,315 cpu_clk_unhalted.thread_any # 54090657.5 CORE_CLKS (66.62%)
1.349436822 79,700,353 cpu_clk_unhalted.thread (66.61%)
1.349436822 3,911,698 cpu_clk_thread_unhalted.one_thread_active # 0.8 SMT_2T_Utilization (66.69%)
1.349436822 14,739,671 cpu_clk_thread_unhalted.ref_xclk_any (66.69%)
could you please check on that and maybe shift the alignment for the longest name?
thanks,
jirka
[toc] | [prev] | [next] | [standalone]
| From | Jiri Olsa <jolsa@redhat.com> |
|---|---|
| Date | 2017-09-01 19:40 +0200 |
| Message-ID | <ukY5c-8en-17@gated-at.bofh.it> |
| In reply to | #1725253 |
On Fri, Sep 01, 2017 at 07:26:18PM +0200, Jiri Olsa wrote: > On Thu, Aug 31, 2017 at 12:40:25PM -0700, Andi Kleen wrote: > > SNIP > > > > > % perf stat -M Summary --metric-only -a sleep 1 > > > > Performance counter stats for 'system wide': > > > > Instructions CLKS CPU_Utilization GFLOPs SMT_2T_Utilization Kernel_Utilization > > 317614222.0 1392930775.0 0.0 0.0 0.2 0.1 > > > > 1.001497549 seconds time elapsed > > > > % perf stat -M GFLOPs flops > > > > Performance counter stats for 'flops': > > > > 3,999,541,471 fp_comp_ops_exe.sse_scalar_single # 1.2 GFLOPs (66.65%) > > 14 fp_comp_ops_exe.sse_scalar_double (66.65%) > > 0 fp_comp_ops_exe.sse_packed_double (66.67%) > > 0 fp_comp_ops_exe.sse_packed_single (66.70%) > > 0 simd_fp_256.packed_double (66.70%) > > 0 simd_fp_256.packed_single (66.67%) > > looks like some events are probably crossing some > output boundaries we have: > > [jolsa@krava perf]$ sudo ./perf stat -M SMT -I 1000 > # time counts unit events > 1.000565706 408,879,985 inst_retired.any # 0.7 CoreIPC (66.68%) > 1.000565706 1,120,999,114 cpu_clk_unhalted.thread_any (66.68%) > 1.000565706 701,285,312 cycles (66.68%) > 1.000565706 1,148,325,740 cpu_clk_unhalted.thread_any # 574162870.0 CORE_CLKS (66.67%) > 1.000565706 711,565,247 cpu_clk_unhalted.thread (66.66%) > 1.000565706 24,057,590 cpu_clk_thread_unhalted.one_thread_active # 0.3 SMT_2T_Utilization (66.67%) > 1.000565706 65,753,475 cpu_clk_thread_unhalted.ref_xclk_any (66.67%) > ^C 1.349436822 21,198,385 inst_retired.any # 0.1 CoreIPC (66.70%) > 1.349436822 112,740,282 cpu_clk_unhalted.thread_any (66.70%) > 1.349436822 84,509,414 cycles (66.70%) > 1.349436822 108,181,315 cpu_clk_unhalted.thread_any # 54090657.5 CORE_CLKS (66.62%) > 1.349436822 79,700,353 cpu_clk_unhalted.thread (66.61%) > 1.349436822 3,911,698 cpu_clk_thread_unhalted.one_thread_active # 0.8 SMT_2T_Utilization (66.69%) > 1.349436822 14,739,671 cpu_clk_thread_unhalted.ref_xclk_any (66.69%) > > > could you please check on that and maybe shift the alignment for the longest name? other than this the rest of the patchset looks ok to me Acked-by: Jiri Olsa <jolsa@kernel.org> thanks, jirka
[toc] | [prev] | [next] | [standalone]
| From | Andi Kleen <andi@firstfloor.org> |
|---|---|
| Date | 2017-09-01 19:50 +0200 |
| Message-ID | <ukYeR-8i9-9@gated-at.bofh.it> |
| In reply to | #1725253 |
> could you please check on that and maybe shift the alignment for the longest name? To make everything align would require shifting everything. So most usages which don't have that wide events wouldn't fit into 80 character columns anymore. That would be worse. Or do a two pass output that computes lengths first. But that's fairly complicated and would prefer to not tackle this right now. Again it's only a few cases where it is visible at all. -Andi
[toc] | [prev] | [next] | [standalone]
| From | Jiri Olsa <jolsa@redhat.com> |
|---|---|
| Date | 2017-09-01 20:00 +0200 |
| Message-ID | <ukYoy-8lL-3@gated-at.bofh.it> |
| In reply to | #1725263 |
On Fri, Sep 01, 2017 at 10:42:04AM -0700, Andi Kleen wrote: > > could you please check on that and maybe shift the alignment for the longest name? > > To make everything align would require shifting everything. So most > usages which don't have that wide events wouldn't fit into 80 > character columns anymore. That would be worse. > > Or do a two pass output that computes lengths first. But that's > fairly complicated and would prefer to not tackle this right now. that's what I had in mind.. but could be dealt with later jirka
[toc] | [prev] | [standalone]
Page 2 of 2 — ← Prev page 1 [2]
Back to top | Article view | linux.kernel
csiph-web