Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1528644 > unrolled thread

[PATCH 00/14] export perf overheads information

Back to article view | Back to linux.kernel

Contents

  [PATCH 00/14] export perf overheads information kan.liang@intel.com - 2016-11-23 18:50 +0100
    [PATCH 01/14] perf/x86: Introduce PERF_RECORD_OVERHEAD kan.liang@intel.com - 2016-11-23 18:50 +0100
      Re: [PATCH 01/14] perf/x86: Introduce PERF_RECORD_OVERHEAD Peter Zijlstra <peterz@infradead.org> - 2016-11-23 21:20 +0100
      Re: [PATCH 01/14] perf/x86: Introduce PERF_RECORD_OVERHEAD Peter Zijlstra <peterz@infradead.org> - 2016-11-23 21:20 +0100
      Re: [PATCH 01/14] perf/x86: Introduce PERF_RECORD_OVERHEAD Jiri Olsa <jolsa@redhat.com> - 2016-11-24 00:50 +0100
        RE: [PATCH 01/14] perf/x86: Introduce PERF_RECORD_OVERHEAD "Liang, Kan" <kan.liang@intel.com> - 2016-11-24 14:50 +0100
          Re: [PATCH 01/14] perf/x86: Introduce PERF_RECORD_OVERHEAD Peter Zijlstra <peterz@infradead.org> - 2016-11-24 15:00 +0100
            RE: [PATCH 01/14] perf/x86: Introduce PERF_RECORD_OVERHEAD "Liang, Kan" <kan.liang@intel.com> - 2016-11-24 15:10 +0100
              Re: [PATCH 01/14] perf/x86: Introduce PERF_RECORD_OVERHEAD Jiri Olsa <jolsa@redhat.com> - 2016-11-24 15:30 +0100
                Re: [PATCH 01/14] perf/x86: Introduce PERF_RECORD_OVERHEAD Jiri Olsa <jolsa@redhat.com> - 2016-11-24 15:50 +0100
                RE: [PATCH 01/14] perf/x86: Introduce PERF_RECORD_OVERHEAD "Liang, Kan" <kan.liang@intel.com> - 2016-11-24 15:50 +0100
            Re: [PATCH 01/14] perf/x86: Introduce PERF_RECORD_OVERHEAD Andi Kleen <andi@firstfloor.org> - 2016-11-24 19:30 +0100
              Re: [PATCH 01/14] perf/x86: Introduce PERF_RECORD_OVERHEAD Peter Zijlstra <peterz@infradead.org> - 2016-11-24 20:00 +0100
                Re: [PATCH 01/14] perf/x86: Introduce PERF_RECORD_OVERHEAD Andi Kleen <andi@firstfloor.org> - 2016-11-24 20:10 +0100
                  Re: [PATCH 01/14] perf/x86: Introduce PERF_RECORD_OVERHEAD Peter Zijlstra <peterz@infradead.org> - 2016-11-24 20:10 +0100
    [PATCH 11/14] perf tools: record write data overhead kan.liang@intel.com - 2016-11-23 18:50 +0100
      Re: [PATCH 11/14] perf tools: record write data overhead Jiri Olsa <jolsa@redhat.com> - 2016-11-24 00:10 +0100
      Re: [PATCH 11/14] perf tools: record write data overhead Jiri Olsa <jolsa@redhat.com> - 2016-11-24 00:20 +0100
    [PATCH 04/14] perf/x86: output side-band events overhead kan.liang@intel.com - 2016-11-23 18:50 +0100
      Re: [PATCH 04/14] perf/x86: output side-band events overhead Peter Zijlstra <peterz@infradead.org> - 2016-11-23 21:10 +0100
      Re: [PATCH 04/14] perf/x86: output side-band events overhead Mark Rutland <mark.rutland@arm.com> - 2016-11-24 17:30 +0100
        RE: [PATCH 04/14] perf/x86: output side-band events overhead "Liang, Kan" <kan.liang@intel.com> - 2016-11-24 20:50 +0100
    [PATCH 07/14] perf tools: show multiplexing overhead kan.liang@intel.com - 2016-11-23 18:50 +0100
    [PATCH 08/14] perf tools: show side-band events overhead kan.liang@intel.com - 2016-11-23 18:50 +0100
    [PATCH 05/14] perf tools: handle PERF_RECORD_OVERHEAD record type kan.liang@intel.com - 2016-11-23 18:50 +0100
      Re: [PATCH 05/14] perf tools: handle PERF_RECORD_OVERHEAD record type Jiri Olsa <jolsa@redhat.com> - 2016-11-23 23:40 +0100
        Re: [PATCH 05/14] perf tools: handle PERF_RECORD_OVERHEAD record type Jiri Olsa <jolsa@redhat.com> - 2016-11-24 00:00 +0100
    [PATCH 12/14] perf tools: record elapsed time kan.liang@intel.com - 2016-11-23 18:50 +0100
    [PATCH 13/14] perf tools: warn on high overhead kan.liang@intel.com - 2016-11-23 18:50 +0100
      Re: [PATCH 13/14] perf tools: warn on high overhead Andi Kleen <andi@firstfloor.org> - 2016-11-23 21:30 +0100
        RE: [PATCH 13/14] perf tools: warn on high overhead "Liang, Kan" <kan.liang@intel.com> - 2016-11-23 23:10 +0100
    [PATCH 14/14] perf script: show overhead events kan.liang@intel.com - 2016-11-23 18:50 +0100
      Re: [PATCH 14/14] perf script: show overhead events Jiri Olsa <jolsa@redhat.com> - 2016-11-24 00:30 +0100
      Re: [PATCH 14/14] perf script: show overhead events Jiri Olsa <jolsa@redhat.com> - 2016-11-24 00:30 +0100
      Re: [PATCH 14/14] perf script: show overhead events Jiri Olsa <jolsa@redhat.com> - 2016-11-24 00:40 +0100
      Re: [PATCH 14/14] perf script: show overhead events Jiri Olsa <jolsa@redhat.com> - 2016-11-24 00:40 +0100
    [PATCH 03/14] perf/x86: output multiplexing overhead kan.liang@intel.com - 2016-11-23 18:50 +0100
      Re: [PATCH 03/14] perf/x86: output multiplexing overhead Peter Zijlstra <peterz@infradead.org> - 2016-11-23 21:10 +0100
        RE: [PATCH 03/14] perf/x86: output multiplexing overhead "Liang, Kan" <kan.liang@intel.com> - 2016-11-23 21:20 +0100
    [PATCH 10/14] perf tools: introduce PERF_RECORD_USER_OVERHEAD kan.liang@intel.com - 2016-11-23 18:50 +0100
    [PATCH 06/14] perf tools: show NMI overhead kan.liang@intel.com - 2016-11-23 18:50 +0100
      Re: [PATCH 06/14] perf tools: show NMI overhead Jiri Olsa <jolsa@redhat.com> - 2016-11-24 00:00 +0100
      Re: [PATCH 06/14] perf tools: show NMI overhead Jiri Olsa <jolsa@redhat.com> - 2016-11-24 00:00 +0100
        RE: [PATCH 06/14] perf tools: show NMI overhead "Liang, Kan" <kan.liang@intel.com> - 2016-11-24 14:40 +0100
          Re: [PATCH 06/14] perf tools: show NMI overhead Jiri Olsa <jolsa@redhat.com> - 2016-11-24 16:30 +0100
            Re: [PATCH 06/14] perf tools: show NMI overhead Namhyung Kim <namhyung@kernel.org> - 2016-11-25 00:30 +0100
              Re: [PATCH 06/14] perf tools: show NMI overhead Jiri Olsa <jolsa@redhat.com> - 2016-11-25 00:50 +0100
            Re: [PATCH 06/14] perf tools: show NMI overhead Andi Kleen <andi@firstfloor.org> - 2016-11-25 01:30 +0100
      Re: [PATCH 06/14] perf tools: show NMI overhead Jiri Olsa <jolsa@redhat.com> - 2016-11-24 00:00 +0100
    Re: [PATCH 00/14] export perf overheads information Ingo Molnar <mingo@kernel.org> - 2016-11-24 05:30 +0100

Page 3 of 3 — ← Prev page 1 2 [3]

#1528657 — [PATCH 06/14] perf tools: show NMI overhead

From: Kan Liang <kan.liang@intel.com>

Caculate the total NMI overhead on each CPU, and display them in perf
report

Signed-off-by: Kan Liang <kan.liang@intel.com>
---
 tools/perf/builtin-report.c | 11 +++++++++++
 tools/perf/util/event.h     |  4 ++++
 tools/perf/util/machine.c   |  9 +++++++++
 tools/perf/util/session.c   | 18 ++++++++++++++++++
 4 files changed, 42 insertions(+)

diff --git a/tools/perf/builtin-report.c b/tools/perf/builtin-report.c
index 1416c39..b1437586 100644
--- a/tools/perf/builtin-report.c
+++ b/tools/perf/builtin-report.c
@@ -365,11 +365,22 @@ static int perf_evlist__tty_browse_hists(struct perf_evlist *evlist,
 					 struct report *rep,
 					 const char *help)
 {
+	struct perf_session *session = rep->session;
 	struct perf_evsel *pos;
+	int cpu;
 
 	fprintf(stdout, "#\n# Total Lost Samples: %" PRIu64 "\n#\n", evlist->stats.total_lost_samples);
 	if (symbol_conf.show_overhead) {
 		fprintf(stdout, "# Overhead:\n");
+		for (cpu = 0; cpu < session->header.env.nr_cpus_online; cpu++) {
+			if (!evlist->stats.total_nmi_overhead[cpu][0])
+				continue;
+			if (rep->cpu_list && !test_bit(cpu, rep->cpu_bitmap))
+				continue;
+			fprintf(stdout, "#\tCPU %d: NMI#: %" PRIu64 " time: %" PRIu64 " ns\n",
+				cpu, evlist->stats.total_nmi_overhead[cpu][0],
+				evlist->stats.total_nmi_overhead[cpu][1]);
+		}
 		fprintf(stdout, "#\n");
 	}
 	evlist__for_each_entry(evlist, pos) {
diff --git a/tools/perf/util/event.h b/tools/perf/util/event.h
index d1b179b..7d40d54 100644
--- a/tools/perf/util/event.h
+++ b/tools/perf/util/event.h
@@ -262,6 +262,9 @@ enum auxtrace_error_type {
  * multipling nr_events[PERF_EVENT_SAMPLE] by a frequency isn't possible to get
  * the total number of low level events, it is necessary to to sum all struct
  * sample_event.period and stash the result in total_period.
+ *
+ * The total_nmi_overhead tells exactly the NMI handler overhead on each CPU.
+ * The total NMI# is stored in [0], while the accumulated time is in [1].
  */
 struct events_stats {
 	u64 total_period;
@@ -270,6 +273,7 @@ struct events_stats {
 	u64 total_lost_samples;
 	u64 total_aux_lost;
 	u64 total_invalid_chains;
+	u64 total_nmi_overhead[MAX_NR_CPUS][2];
 	u32 nr_events[PERF_RECORD_HEADER_MAX];
 	u32 nr_non_filtered_samples;
 	u32 nr_lost_warned;
diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c
index 1101757..58076f2 100644
--- a/tools/perf/util/machine.c
+++ b/tools/perf/util/machine.c
@@ -558,6 +558,15 @@ int machine__process_switch_event(struct machine *machine __maybe_unused,
 int machine__process_overhead_event(struct machine *machine __maybe_unused,
 				    union perf_event *event __maybe_unused)
 {
+	if (event->overhead.type == PERF_NMI_OVERHEAD) {
+		dump_printf(" NMI nr: %llu  time: %llu cpu %u\n",
+			    event->overhead.entry.nr,
+			    event->overhead.entry.time,
+			    event->overhead.entry.cpu);
+	} else {
+		dump_printf("\tUNSUPPORT OVERHEAD TYPE 0x%x!\n", event->overhead.type);
+	}
+
 	return 0;
 }
 
diff --git a/tools/perf/util/session.c b/tools/perf/util/session.c
index bc0bc21..a79ab99 100644
--- a/tools/perf/util/session.c
+++ b/tools/perf/util/session.c
@@ -1207,6 +1207,23 @@ static int
 					    &sample->read.one, machine);
 }
 
+static void
+overhead_stats_update(struct perf_tool *tool,
+		      struct perf_evlist *evlist,
+		      union perf_event *event)
+{
+	if (tool->overhead == perf_event__process_overhead) {
+		switch (event->overhead.type) {
+		case PERF_NMI_OVERHEAD:
+			evlist->stats.total_nmi_overhead[event->overhead.entry.cpu][0] += event->overhead.entry.nr;
+			evlist->stats.total_nmi_overhead[event->overhead.entry.cpu][1] += event->overhead.entry.time;
+			break;
+		default:
+			break;
+		}
+	}
+}
+
 static int machines__deliver_event(struct machines *machines,
 				   struct perf_evlist *evlist,
 				   union perf_event *event,
@@ -1271,6 +1288,7 @@ static int machines__deliver_event(struct machines *machines,
 	case PERF_RECORD_SWITCH_CPU_WIDE:
 		return tool->context_switch(tool, event, sample, machine);
 	case PERF_RECORD_OVERHEAD:
+		overhead_stats_update(tool, evlist, event);
 		return tool->overhead(tool, event, sample, machine);
 	default:
 		++evlist->stats.nr_unknown_events;
-- 
2.5.5

[toc] | [prev] | [next] | [standalone]

#1528824 — Re: [PATCH 06/14] perf tools: show NMI overhead

On Wed, Nov 23, 2016 at 04:44:44AM -0500, kan.liang@intel.com wrote:
> From: Kan Liang <kan.liang@intel.com>
> 
> Caculate the total NMI overhead on each CPU, and display them in perf
> report

please put example output into chagelog

thanks,
jirka

[toc] | [prev] | [next] | [standalone]

#1528826 — Re: [PATCH 06/14] perf tools: show NMI overhead

On Wed, Nov 23, 2016 at 04:44:44AM -0500, kan.liang@intel.com wrote:
> From: Kan Liang <kan.liang@intel.com>
> 
> Caculate the total NMI overhead on each CPU, and display them in perf
> report

so the output looks like this:

---
# Elapsed time: 1720167944 ns
# Overhead:
#       CPU 6
#               NMI#: 27 time: 111379 ns
#               Multiplexing#: 0 time: 0 ns
#               SB#: 57 time: 90045 ns
#
# Samples: 26  of event 'cycles:u'
# Event count (approx.): 1677531
#
# Overhead  Command  Shared Object     Symbol                 
# ........  .......  ................  .......................
#
    24.20%  ls       ls                [.] _init
    17.18%  ls       libc-2.24.so      [.] __strcoll_l
    11.85%  ls       ld-2.24.so        [.] _dl_relocate_object
---


few things:

- I wonder we want to put this overhead output separatelly from the
  main perf out.. this scale bad with with bigger cpu counts

- we might want to call it some other way, becayse we already
  use 'overhead' for the event count %

- how about TUI output? ;-) I dont think it's necessary, however
  currently 'perf report --show-overhead' does not show anything
  ifTUI is default output, unless you use --stdio option

thanks,
jirka

[toc] | [prev] | [next] | [standalone]

#1529286 — RE: [PATCH 06/14] perf tools: show NMI overhead

> 
> On Wed, Nov 23, 2016 at 04:44:44AM -0500, kan.liang@intel.com wrote:
> > From: Kan Liang <kan.liang@intel.com>
> >
> > Caculate the total NMI overhead on each CPU, and display them in perf
> > report
> 
> so the output looks like this:
> 
> ---
> # Elapsed time: 1720167944 ns
> # Overhead:
> #       CPU 6
> #               NMI#: 27 time: 111379 ns
> #               Multiplexing#: 0 time: 0 ns
> #               SB#: 57 time: 90045 ns
> #
> # Samples: 26  of event 'cycles:u'
> # Event count (approx.): 1677531
> #
> # Overhead  Command  Shared Object     Symbol
> # ........  .......  ................  .......................
> #
>     24.20%  ls       ls                [.] _init
>     17.18%  ls       libc-2.24.so      [.] __strcoll_l
>     11.85%  ls       ld-2.24.so        [.] _dl_relocate_object
> ---
> 
> 
> few things:
> 
> - I wonder we want to put this overhead output separatelly from the
>   main perf out.. this scale bad with with bigger cpu counts
> 
This output can only be shown when the user explicitly apply
the --show-overhead option. I think the user should expect the big
header.
Or I can add  --show-overhead-only option which only show the
overhead information. It will like what we do for --header and
--header-only 

Any suggestions?

> - we might want to call it some other way, becayse we already
>   use 'overhead' for the event count %
>

"operating_cost"? "processing_cost"? "perf_cost"? "perf_overhead"?
Suggestions?
 
> - how about TUI output? ;-) I dont think it's necessary, however
>   currently 'perf report --show-overhead' does not show anything
>   ifTUI is default output, unless you use --stdio option

I will try to add something in TUI mode.

Thanks,
Kan

[toc] | [prev] | [next] | [standalone]

#1529435 — Re: [PATCH 06/14] perf tools: show NMI overhead

On Thu, Nov 24, 2016 at 01:37:04PM +0000, Liang, Kan wrote:
> 
> 
> > 
> > On Wed, Nov 23, 2016 at 04:44:44AM -0500, kan.liang@intel.com wrote:
> > > From: Kan Liang <kan.liang@intel.com>
> > >
> > > Caculate the total NMI overhead on each CPU, and display them in perf
> > > report
> > 
> > so the output looks like this:
> > 
> > ---
> > # Elapsed time: 1720167944 ns
> > # Overhead:
> > #       CPU 6
> > #               NMI#: 27 time: 111379 ns
> > #               Multiplexing#: 0 time: 0 ns
> > #               SB#: 57 time: 90045 ns
> > #
> > # Samples: 26  of event 'cycles:u'
> > # Event count (approx.): 1677531
> > #
> > # Overhead  Command  Shared Object     Symbol
> > # ........  .......  ................  .......................
> > #
> >     24.20%  ls       ls                [.] _init
> >     17.18%  ls       libc-2.24.so      [.] __strcoll_l
> >     11.85%  ls       ld-2.24.so        [.] _dl_relocate_object
> > ---

how about we display the overhead information same way the main perf output:

  CPU    NMI   NMI time    MTX  MTX time      SB   SB time
  ...  .....   ........  .....  ........  ......  ........
    6     27     111379      0         0      57     90045


would be just matter of adding new sort objects

jirka

[toc] | [prev] | [next] | [standalone]

#1529738 — Re: [PATCH 06/14] perf tools: show NMI overhead

Hi,

On Thu, Nov 24, 2016 at 04:27:21PM +0100, Jiri Olsa wrote:
> On Thu, Nov 24, 2016 at 01:37:04PM +0000, Liang, Kan wrote:
> > 
> > 
> > > 
> > > On Wed, Nov 23, 2016 at 04:44:44AM -0500, kan.liang@intel.com wrote:
> > > > From: Kan Liang <kan.liang@intel.com>
> > > >
> > > > Caculate the total NMI overhead on each CPU, and display them in perf
> > > > report
> > > 
> > > so the output looks like this:
> > > 
> > > ---
> > > # Elapsed time: 1720167944 ns
> > > # Overhead:
> > > #       CPU 6
> > > #               NMI#: 27 time: 111379 ns
> > > #               Multiplexing#: 0 time: 0 ns
> > > #               SB#: 57 time: 90045 ns
> > > #
> > > # Samples: 26  of event 'cycles:u'
> > > # Event count (approx.): 1677531
> > > #
> > > # Overhead  Command  Shared Object     Symbol
> > > # ........  .......  ................  .......................
> > > #
> > >     24.20%  ls       ls                [.] _init
> > >     17.18%  ls       libc-2.24.so      [.] __strcoll_l
> > >     11.85%  ls       ld-2.24.so        [.] _dl_relocate_object
> > > ---
> 
> how about we display the overhead information same way the main perf output:
> 
>   CPU    NMI   NMI time    MTX  MTX time      SB   SB time
>   ...  .....   ........  .....  ........  ......  ........
>     6     27     111379      0         0      57     90045
> 
> 
> would be just matter of adding new sort objects

How would you connect those to hist entries then?  It'd be possible if
the sort key had 'cpu' only, no?

Thanks,
Namhyung

[toc] | [prev] | [next] | [standalone]

#1529742 — Re: [PATCH 06/14] perf tools: show NMI overhead

On Fri, Nov 25, 2016 at 08:20:13AM +0900, Namhyung Kim wrote:
> Hi,
> 
> On Thu, Nov 24, 2016 at 04:27:21PM +0100, Jiri Olsa wrote:
> > On Thu, Nov 24, 2016 at 01:37:04PM +0000, Liang, Kan wrote:
> > > 
> > > 
> > > > 
> > > > On Wed, Nov 23, 2016 at 04:44:44AM -0500, kan.liang@intel.com wrote:
> > > > > From: Kan Liang <kan.liang@intel.com>
> > > > >
> > > > > Caculate the total NMI overhead on each CPU, and display them in perf
> > > > > report
> > > > 
> > > > so the output looks like this:
> > > > 
> > > > ---
> > > > # Elapsed time: 1720167944 ns
> > > > # Overhead:
> > > > #       CPU 6
> > > > #               NMI#: 27 time: 111379 ns
> > > > #               Multiplexing#: 0 time: 0 ns
> > > > #               SB#: 57 time: 90045 ns
> > > > #
> > > > # Samples: 26  of event 'cycles:u'
> > > > # Event count (approx.): 1677531
> > > > #
> > > > # Overhead  Command  Shared Object     Symbol
> > > > # ........  .......  ................  .......................
> > > > #
> > > >     24.20%  ls       ls                [.] _init
> > > >     17.18%  ls       libc-2.24.so      [.] __strcoll_l
> > > >     11.85%  ls       ld-2.24.so        [.] _dl_relocate_object
> > > > ---
> > 
> > how about we display the overhead information same way the main perf output:
> > 
> >   CPU    NMI   NMI time    MTX  MTX time      SB   SB time
> >   ...  .....   ........  .....  ........  ......  ........
> >     6     27     111379      0         0      57     90045
> > 
> > 
> > would be just matter of adding new sort objects
> 
> How would you connect those to hist entries then?  It'd be possible if
> the sort key had 'cpu' only, no?

right, I should have said fields then..

jirka

[toc] | [prev] | [next] | [standalone]

#1529753 — Re: [PATCH 06/14] perf tools: show NMI overhead

> how about we display the overhead information same way the main perf output:
> 
>   CPU    NMI   NMI time    MTX  MTX time      SB   SB time
>   ...  .....   ........  .....  ........  ......  ........
>     6     27     111379      0         0      57     90045
> 
> 
> would be just matter of adding new sort objects

The problem with making overhead a standard sort key is that you have
to chose between an output format that makes sense for overhead
and one that makes sense for normal samples.

But overhead is more "auxillary" information, so it should be possible
to access it together with normal sampling information in a single
output file.

So I think it's better handled separately.

-Andi

[toc] | [prev] | [next] | [standalone]

#1528830 — Re: [PATCH 06/14] perf tools: show NMI overhead

On Wed, Nov 23, 2016 at 04:44:44AM -0500, kan.liang@intel.com wrote:
> From: Kan Liang <kan.liang@intel.com>
> 
> Caculate the total NMI overhead on each CPU, and display them in perf
> report
> 
> Signed-off-by: Kan Liang <kan.liang@intel.com>
> ---
>  tools/perf/builtin-report.c | 11 +++++++++++
>  tools/perf/util/event.h     |  4 ++++
>  tools/perf/util/machine.c   |  9 +++++++++
>  tools/perf/util/session.c   | 18 ++++++++++++++++++
>  4 files changed, 42 insertions(+)
> 
> diff --git a/tools/perf/builtin-report.c b/tools/perf/builtin-report.c
> index 1416c39..b1437586 100644
> --- a/tools/perf/builtin-report.c
> +++ b/tools/perf/builtin-report.c
> @@ -365,11 +365,22 @@ static int perf_evlist__tty_browse_hists(struct perf_evlist *evlist,
>  					 struct report *rep,
>  					 const char *help)
>  {
> +	struct perf_session *session = rep->session;
>  	struct perf_evsel *pos;
> +	int cpu;
>  
>  	fprintf(stdout, "#\n# Total Lost Samples: %" PRIu64 "\n#\n", evlist->stats.total_lost_samples);
>  	if (symbol_conf.show_overhead) {
>  		fprintf(stdout, "# Overhead:\n");
> +		for (cpu = 0; cpu < session->header.env.nr_cpus_online; cpu++) {
> +			if (!evlist->stats.total_nmi_overhead[cpu][0])
> +				continue;
> +			if (rep->cpu_list && !test_bit(cpu, rep->cpu_bitmap))
> +				continue;
> +			fprintf(stdout, "#\tCPU %d: NMI#: %" PRIu64 " time: %" PRIu64 " ns\n",
> +				cpu, evlist->stats.total_nmi_overhead[cpu][0],
> +				evlist->stats.total_nmi_overhead[cpu][1]);
> +		}
>  		fprintf(stdout, "#\n");
>  	}
>  	evlist__for_each_entry(evlist, pos) {
> diff --git a/tools/perf/util/event.h b/tools/perf/util/event.h
> index d1b179b..7d40d54 100644
> --- a/tools/perf/util/event.h
> +++ b/tools/perf/util/event.h
> @@ -262,6 +262,9 @@ enum auxtrace_error_type {
>   * multipling nr_events[PERF_EVENT_SAMPLE] by a frequency isn't possible to get
>   * the total number of low level events, it is necessary to to sum all struct
>   * sample_event.period and stash the result in total_period.
> + *
> + * The total_nmi_overhead tells exactly the NMI handler overhead on each CPU.
> + * The total NMI# is stored in [0], while the accumulated time is in [1].
>   */

hum, why can't this be stored this in the struct instead.. ?

thanks,
jirka

[toc] | [prev] | [next] | [standalone]

#1528967

* kan.liang@intel.com <kan.liang@intel.com> wrote:

> From: Kan Liang <kan.liang@intel.com>
> 
> Profiling brings additional overhead. High overhead may impacts the
> behavior of the profiling object, impacts the accuracy of the
> profiling result, and even hang the system.
> Currently, perf has dynamic interrupt throttle mechanism to lower the
> sample rate and overhead. But it has limitations.
>  - The mechanism only focus in the overhead from NMI. However, there
>    are other parts which bring big overhead. E.g, multiplexing.
>  - The hint from the mechanism doesn't work on fixed period.
>  - The system changes which caused by the mechanism are not recorded
>    in the perf.data. Users have no idea about the overhead and its
>    impact.
> Acctually, any passive ways like dynamic interrupt throttle mechanism
> are only palliative. The best way is to export overheads information,
> provide more hints, and help the users design more proper perf command.
> 
> According to our test, there are four parts which can bring big overhead.
> They include NMI handler, multiplexing handler, iterate side-band events,
> and write data in file. Two new perf record type PERF_RECORD_OVERHEAD and
> PERF_RECORD_USER_OVERHEAD are introduced to record the overhead
> information in kernel and user space respectively.
> The overhead information is the system per-CPU overhead, not per-event
> overhead. The implementation takes advantage of the existing event log
> mechanism.
> To reduce the additional overhead from logging overhead information, the
> overhead information only be output when the event is going to be
> disabled or task is scheduling out.
> 
> In perf report, the overhead will be checked automatically. If the
> overhead rate is larger than 10%. A warning will be displayed.
> A new option is also introduced to display detial per-CPU overhead
> information.
> 
> Current implementation only include four overhead sources. There could be
> more in other parts. The new overhead source can be easily added as a
> new type.

Please include sample output of the new instrumentation!

Not even the tooling patches show any of the output, nor is it clear anywhere what 
kind of 'overhead' measurement it is, what the units are, what the metrics are, 
how users can _use_ this information, etc.

This is totally inadequate description.

Thanks,

	Ingo

[toc] | [prev] | [standalone]

Page 3 of 3 — ← Prev page 1 2 [3]

Back to top | Article view | linux.kernel

csiph-web