Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1230978 > unrolled thread

[PATCH RFC 00/10] counter read during perf sampling

Back to article view | Back to linux.kernel

Contents

  [PATCH RFC 00/10] counter read during perf sampling kan.liang@intel.com - 2015-09-22 23:40 +0200
    [PATCH RFC 01/10] perf,tools: Add 'C' event/group modifier kan.liang@intel.com - 2015-09-22 23:40 +0200
    [PATCH RFC 02/10] perf,tools: Enable counter statistic read for perf record kan.liang@intel.com - 2015-09-22 23:40 +0200
    Re: [PATCH RFC 00/10] counter read during perf sampling Jiri Olsa <jolsa@redhat.com> - 2015-09-24 10:20 +0200
      RE: [PATCH RFC 00/10] counter read during perf sampling "Liang, Kan" <kan.liang@intel.com> - 2015-09-24 21:50 +0200
        Re: [PATCH RFC 00/10] counter read during perf sampling Jiri Olsa <jolsa@redhat.com> - 2015-09-25 00:30 +0200
          RE: [PATCH RFC 00/10] counter read during perf sampling "Liang, Kan" <kan.liang@intel.com> - 2015-09-25 17:00 +0200
            Re: [PATCH RFC 00/10] counter read during perf sampling Jiri Olsa <jolsa@redhat.com> - 2015-09-27 22:00 +0200
              RE: [PATCH RFC 00/10] counter read during perf sampling "Liang, Kan" <kan.liang@intel.com> - 2015-09-28 17:20 +0200

#1230978 — [PATCH RFC 00/10] counter read during perf sampling

From: Kan Liang <kan.liang@intel.com>

The patch series intends to read counter statistics with fixed frequency
during sampling. The instant benefit is that we can read memory bandwidth
from uncore event during cpu PMU event is sampling.

Introduce 'C' event/group modifier. The event with this modifier
will do counting not sampling. If a group with this modifier, only
group leader do sampling. The counter statistics will be wrote in
new RECORD type PERF_RECORD_COUNTER_READ and stored in perf.data.
So perf report can present the counter statistics data accordingly.

There may be an alternative way to get counter statistics during
sampling by running perf record and perf stat together by script.
But the script way have various issue and complex to parses the
output.

Example:

 $perf record -e 'cycles,uncore_imc_1/cas_count_read/C'
  --counter-read-interval 10 -a ./tchain_edit
 [ perf record: Woken up 438 times to write data ]
 [ perf record: Captured and wrote 1.232 MB perf.data (17901 samples) ]


 $perf report -D

 0x3cae0 [0x28]: PERF_RECORD_COUNTER_READ: uncore_imc_1/cas_count_read/C
                CPU 0: val 1205 ena 2046148 run 2046148

 0x3cb08 [0x28]: PERF_RECORD_COUNTER_READ: uncore_imc_1/cas_count_read/C
                CPU 18: val 1315 ena 2001918 run 2001918

 0x3dba0 [0x28]: PERF_RECORD_COUNTER_READ: uncore_imc_1/cas_count_read/C
                CPU 0: val 1588 ena 12191520 run 12191520

 0x3dbc8 [0x28]: PERF_RECORD_COUNTER_READ: uncore_imc_1/cas_count_read/C
                CPU 18: val 1686 ena 12162202 run 12162202

 $perf report --stdio --socket-filter 0

 # To display the perf.data header info, please use --header/--header-only options.
 #
 #
 # Total Lost Samples: 0
 #
 # Samples: 17K of event 'cycles'
 # Event count (approx.): 10119095556
 # Processor Socket: 0
 #
 # Overhead  Command       Shared Object        Symbol
 # ........  ............  ...................  ..................................
 #
    97.68%  tchain_edit   tchain_edit          [.] f3
     0.07%  tchain_edit   tchain_edit          [.] f2
     0.04%  swapper       [kernel.vmlinux]     [k] run_timer_softirq
     0.04%  swapper       [kernel.vmlinux]     [k] find_busiest_group

 # Samples: 0  of event 'uncore_imc_1/cas_count_read/C'
 # Event count (approx.): 0
 # Processor Socket: 0
 # uncore_imc_1/cas_count_read/C: 35937


Kan Liang (10):
  perf,tools: Add 'C' event/group modifier
  perf,tools: Enable counter statistic read for perf record
  perf,tools: don't validate counter read event
  perf,tools: New RECORD type PERF_RECORD_COUNTER_READ
  perf,tools: record counter statistics during sampling
  perf,tools: option to set counter read interval
  perf,report: handle PERF_RECORD_COUNTER_READ
  perf,tools: store counter val in events_stats
  perf,tools: show counter read result in studio
  perf,tools: show counter read result in tui browser title

 tools/perf/Documentation/perf-list.txt   |   5 ++
 tools/perf/Documentation/perf-record.txt |   8 +++
 tools/perf/builtin-record.c              |  73 ++++++++++++++++++++++
 tools/perf/builtin-report.c              | 101 ++++++++++++++++++++++++++++++-
 tools/perf/ui/browsers/hists.c           |  37 +++++++++++
 tools/perf/util/event.c                  |   1 +
 tools/perf/util/event.h                  |  21 +++++++
 tools/perf/util/evlist.c                 |  20 ++++++
 tools/perf/util/evsel.c                  |  29 +++++++++
 tools/perf/util/evsel.h                  |   1 +
 tools/perf/util/parse-events.c           |   8 ++-
 tools/perf/util/parse-events.l           |   2 +-
 tools/perf/util/record.c                 |   6 +-
 tools/perf/util/session.c                |  16 +++++
 tools/perf/util/tool.h                   |   1 +
 15 files changed, 324 insertions(+), 5 deletions(-)

-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1230979 — [PATCH RFC 01/10] perf,tools: Add 'C' event/group modifier

From: Kan Liang <kan.liang@intel.com>

Add a new event/group modifier 'C' to mark the event which will be read
counter statistics during sampling.

Signed-off-by: Kan Liang <kan.liang@intel.com>
---
 tools/perf/util/evsel.h        | 1 +
 tools/perf/util/parse-events.c | 8 +++++++-
 tools/perf/util/parse-events.l | 2 +-
 3 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h
index 7906666..d017795 100644
--- a/tools/perf/util/evsel.h
+++ b/tools/perf/util/evsel.h
@@ -115,6 +115,7 @@ struct perf_evsel {
 	int			exclude_GH;
 	int			nr_members;
 	int			sample_read;
+	int			counter_read;
 	unsigned long		*per_pkg_mask;
 	struct perf_evsel	*leader;
 	char			*group_name;
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index 61c2bc2..95b43a4 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -860,6 +860,7 @@ struct event_modifier {
 	int precise;
 	int exclude_GH;
 	int sample_read;
+	int counter_read;
 	int pinned;
 };
 
@@ -874,6 +875,7 @@ static int get_event_modifier(struct event_modifier *mod, char *str,
 	int eI = evsel ? evsel->attr.exclude_idle : 0;
 	int precise = evsel ? evsel->attr.precise_ip : 0;
 	int sample_read = 0;
+	int counter_read = 0;
 	int pinned = evsel ? evsel->attr.pinned : 0;
 
 	int exclude = eu | ek | eh;
@@ -911,6 +913,8 @@ static int get_event_modifier(struct event_modifier *mod, char *str,
 				eG = 1;
 		} else if (*str == 'S') {
 			sample_read = 1;
+		} else if (*str == 'C') {
+			counter_read = 1;
 		} else if (*str == 'D') {
 			pinned = 1;
 		} else
@@ -941,6 +945,7 @@ static int get_event_modifier(struct event_modifier *mod, char *str,
 	mod->precise = precise;
 	mod->exclude_GH = exclude_GH;
 	mod->sample_read = sample_read;
+	mod->counter_read = counter_read;
 	mod->pinned = pinned;
 
 	return 0;
@@ -955,7 +960,7 @@ static int check_modifier(char *str)
 	char *p = str;
 
 	/* The sizeof includes 0 byte as well. */
-	if (strlen(str) > (sizeof("ukhGHpppSDI") - 1))
+	if (strlen(str) > (sizeof("ukhGHpppSCDI") - 1))
 		return -1;
 
 	while (*p) {
@@ -994,6 +999,7 @@ int parse_events__modifier_event(struct list_head *list, char *str, bool add)
 		evsel->attr.exclude_idle   = mod.eI;
 		evsel->exclude_GH          = mod.exclude_GH;
 		evsel->sample_read         = mod.sample_read;
+		evsel->counter_read           = mod.counter_read;
 
 		if (perf_evsel__is_group_leader(evsel))
 			evsel->attr.pinned = mod.pinned;
diff --git a/tools/perf/util/parse-events.l b/tools/perf/util/parse-events.l
index 936d566..e6c5790 100644
--- a/tools/perf/util/parse-events.l
+++ b/tools/perf/util/parse-events.l
@@ -122,7 +122,7 @@ num_raw_hex	[a-fA-F0-9]+
 name		[a-zA-Z_*?][a-zA-Z0-9_*?.]*
 name_minus	[a-zA-Z_*?][a-zA-Z0-9\-_*?.]*
 /* If you add a modifier you need to update check_modifier() */
-modifier_event	[ukhpGHSDI]+
+modifier_event	[ukhpGHSCDI]+
 modifier_bp	[rwx]{1,3}
 
 %%
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1230980 — [PATCH RFC 02/10] perf,tools: Enable counter statistic read for perf record

From: Kan Liang <kan.liang@intel.com>

Using 'C' event/group modifier to specify the event which want to read
counter statistics during sampling. For this event, the sampling will
be disabled.
The 'C' modifier only be available on system-wide/CPU mode. If a
group is marked as 'C' modifier, only group members read counter
statistics. Group leader always do sampling.
The other limit is that the first event cannot be counter statistics
read event, since many tools special handle first event.

Signed-off-by: Kan Liang <kan.liang@intel.com>
---
 tools/perf/Documentation/perf-list.txt |  5 +++++
 tools/perf/util/evlist.c               |  3 +++
 tools/perf/util/evsel.c                | 29 +++++++++++++++++++++++++++++
 tools/perf/util/record.c               |  6 ++++--
 4 files changed, 41 insertions(+), 2 deletions(-)

diff --git a/tools/perf/Documentation/perf-list.txt b/tools/perf/Documentation/perf-list.txt
index bada893..a409fc9 100644
--- a/tools/perf/Documentation/perf-list.txt
+++ b/tools/perf/Documentation/perf-list.txt
@@ -31,6 +31,11 @@ counted. The following modifiers exist:
  H - host counting (not in KVM guests)
  p - precise level
  S - read sample value (PERF_SAMPLE_READ)
+ C - read counter statistics during sampling (Can be used to read
+     counter statistics of PMU_A event when PMU_B events are sampling.
+     For example, getting memory bandwidth by uncore events during the
+     CPU PMU events run time)
+     (Only available for group members or non-first event in system-wide/CPU mode)
  D - pin the event to the PMU
 
 The 'p' modifier can be used for specifying how precise the instruction
diff --git a/tools/perf/util/evlist.c b/tools/perf/util/evlist.c
index a864373..603ee3e 100644
--- a/tools/perf/util/evlist.c
+++ b/tools/perf/util/evlist.c
@@ -885,6 +885,9 @@ static int perf_evlist__mmap_per_evsel(struct perf_evlist *evlist, int idx,
 		if (evsel->system_wide && thread)
 			continue;
 
+		if (evsel->counter_read)
+			continue;
+
 		fd = FD(evsel, cpu, thread);
 
 		if (*output == -1) {
diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
index 5889004..8c18422 100644
--- a/tools/perf/util/evsel.c
+++ b/tools/perf/util/evsel.c
@@ -724,6 +724,7 @@ void perf_evsel__config(struct perf_evsel *evsel, struct record_opts *opts)
 	struct perf_event_attr *attr = &evsel->attr;
 	int track = evsel->tracking;
 	bool per_cpu = opts->target.default_per_cpu && !opts->target.per_thread;
+	struct perf_evsel *first = perf_evlist__first(evsel->evlist);
 
 	attr->sample_id_all = perf_missing_features.sample_id_all ? 0 : 1;
 	attr->inherit	    = !opts->no_inherit;
@@ -882,6 +883,34 @@ void perf_evsel__config(struct perf_evsel *evsel, struct record_opts *opts)
 		attr->clockid = opts->clockid;
 	}
 
+	if (evsel->counter_read) {
+		if (!target__has_cpu(&opts->target)) {
+			evsel->counter_read = 0;
+			ui__warning("Counter statistics read only available "
+				    "on system-wide/CPU mode.\n"
+				    "Remove :C modifier for event %s\n",
+				    evsel->name);
+		} else {
+			/* Don't do counter read for Group leader */
+			if ((evsel->leader == evsel) && (evsel->leader->nr_members > 1)) {
+				evsel->counter_read = 0;
+			} else {
+				if (first == evsel) {
+					evsel->counter_read = 0;
+					ui__warning("The first event cannot be counter read event"
+						    "Remove :C modifier for event %s\n",
+						    evsel->name);
+				} else {
+					attr->read_format = PERF_FORMAT_TOTAL_TIME_ENABLED |
+							    PERF_FORMAT_TOTAL_TIME_RUNNING;
+					attr->sample_freq = 0;
+					attr->sample_period = 0;
+					attr->sample_type = 0;
+					evsel->sample_size = 0;
+				}
+			}
+		}
+	}
 	/*
 	 * Apply event specific term settings,
 	 * it overloads any global configuration.
diff --git a/tools/perf/util/record.c b/tools/perf/util/record.c
index 0467367..9ede1f7 100644
--- a/tools/perf/util/record.c
+++ b/tools/perf/util/record.c
@@ -171,8 +171,10 @@ void perf_evlist__config(struct perf_evlist *evlist, struct record_opts *opts)
 			use_sample_identifier = perf_can_sample_identifier();
 			break;
 		}
-		evlist__for_each(evlist, evsel)
-			perf_evsel__set_sample_id(evsel, use_sample_identifier);
+		evlist__for_each(evlist, evsel) {
+			if (!evsel->counter_read)
+				perf_evsel__set_sample_id(evsel, use_sample_identifier);
+		}
 	}
 
 	perf_evlist__set_id_pos(evlist);
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1231935

On Tue, Sep 22, 2015 at 10:13:33AM -0400, kan.liang@intel.com wrote:
> From: Kan Liang <kan.liang@intel.com>
> 
> The patch series intends to read counter statistics with fixed frequency
> during sampling. The instant benefit is that we can read memory bandwidth
> from uncore event during cpu PMU event is sampling.
> 
> Introduce 'C' event/group modifier. The event with this modifier
> will do counting not sampling. If a group with this modifier, only
> group leader do sampling. The counter statistics will be wrote in
> new RECORD type PERF_RECORD_COUNTER_READ and stored in perf.data.
> So perf report can present the counter statistics data accordingly.
> 
> There may be an alternative way to get counter statistics during
> sampling by running perf record and perf stat together by script.
> But the script way have various issue and complex to parses the
> output.

just a thought, but isn't the way then llow to store the data from perf stat? ;-)
and be able to merge perf.data-s from perf record and stat afterwards

> 
> Example:
> 
>  $perf record -e 'cycles,uncore_imc_1/cas_count_read/C'
>   --counter-read-interval 10 -a ./tchain_edit
>  [ perf record: Woken up 438 times to write data ]
>  [ perf record: Captured and wrote 1.232 MB perf.data (17901 samples) ]

but if we go this way I think we should keep/allow all the options perf stat

something like:
  $ perf record -e cycles stat -e 'uncore_imc_1/cas_count_read/' -I 10000 -a ./tchain_edit

with all the stat option we allow -c -F ... 
and reusing existing stat code

>  $perf report -D
> 
>  0x3cae0 [0x28]: PERF_RECORD_COUNTER_READ: uncore_imc_1/cas_count_read/C
>                 CPU 0: val 1205 ena 2046148 run 2046148
> 
>  0x3cb08 [0x28]: PERF_RECORD_COUNTER_READ: uncore_imc_1/cas_count_read/C
>                 CPU 18: val 1315 ena 2001918 run 2001918
> 
>  0x3dba0 [0x28]: PERF_RECORD_COUNTER_READ: uncore_imc_1/cas_count_read/C
>                 CPU 0: val 1588 ena 12191520 run 12191520
> 
>  0x3dbc8 [0x28]: PERF_RECORD_COUNTER_READ: uncore_imc_1/cas_count_read/C
>                 CPU 18: val 1686 ena 12162202 run 12162202
> 
>  $perf report --stdio --socket-filter 0
> 
>  # To display the perf.data header info, please use --header/--header-only options.
>  #
>  #
>  # Total Lost Samples: 0
>  #
>  # Samples: 17K of event 'cycles'
>  # Event count (approx.): 10119095556
>  # Processor Socket: 0
>  #
>  # Overhead  Command       Shared Object        Symbol
>  # ........  ............  ...................  ..................................
>  #
>     97.68%  tchain_edit   tchain_edit          [.] f3
>      0.07%  tchain_edit   tchain_edit          [.] f2
>      0.04%  swapper       [kernel.vmlinux]     [k] run_timer_softirq
>      0.04%  swapper       [kernel.vmlinux]     [k] find_busiest_group
> 
>  # Samples: 0  of event 'uncore_imc_1/cas_count_read/C'
>  # Event count (approx.): 0
>  # Processor Socket: 0
>  # uncore_imc_1/cas_count_read/C: 35937

I think we'll need special output/display for non sampling events,
something like extra window in TUI and distinguished output in stdio,
the above is hacked sampling output ;-)

thoughts?
jirka
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1232395

> On Tue, Sep 22, 2015 at 10:13:33AM -0400, kan.liang@intel.com wrote:
> > From: Kan Liang <kan.liang@intel.com>
> >
> > The patch series intends to read counter statistics with fixed
> > frequency during sampling. The instant benefit is that we can read
> > memory bandwidth from uncore event during cpu PMU event is
> sampling.
> >
> > Introduce 'C' event/group modifier. The event with this modifier will
> > do counting not sampling. If a group with this modifier, only group
> > leader do sampling. The counter statistics will be wrote in new RECORD
> > type PERF_RECORD_COUNTER_READ and stored in perf.data.
> > So perf report can present the counter statistics data accordingly.
> >
> > There may be an alternative way to get counter statistics during
> > sampling by running perf record and perf stat together by script.
> > But the script way have various issue and complex to parses the
> > output.
> 
> just a thought, but isn't the way then llow to store the data from perf
> stat? ;-) and be able to merge perf.data-s from perf record and stat
> afterwards

Yes, the way to store the data from perf stat is better than pure script
way. I guess your patch "perf stat record" can do that, right?

If so, how should we run perf record and stat in parallel? By scripts or
modify perf record/stat? 

Also, we need an option in perf report to merge the perf.data-s. Right?

> 
> >
> > Example:
> >
> >  $perf record -e 'cycles,uncore_imc_1/cas_count_read/C'
> >   --counter-read-interval 10 -a ./tchain_edit  [ perf record: Woken up
> > 438 times to write data ]  [ perf record: Captured and wrote 1.232 MB
> > perf.data (17901 samples) ]
> 
> but if we go this way I think we should keep/allow all the options perf stat

Do you mean something like "perf record stat"?
That's not the way I designed. I don't want to run perf record and perf stat
together in one command.

I just want to do similar thing like what sample read did. Sample read can read
counters on each sample. While the counter read can read counters in a fix
frequency (set by --counter-read-interval). So it's an extension of perf record.
It applies all possible options of perf record, like -C -a -g... 
I introduce a new option --counter-read-interval is because that there is no
 interval options in perf record.

> 
> something like:
>   $ perf record -e cycles stat -e 'uncore_imc_1/cas_count_read/' -I 10000 -
> a ./tchain_edit
> 
> with all the stat option we allow -c -F ...
> and reusing existing stat code
> 
> >  $perf report -D
> >
> >  0x3cae0 [0x28]: PERF_RECORD_COUNTER_READ:
> uncore_imc_1/cas_count_read/C
> >                 CPU 0: val 1205 ena 2046148 run 2046148
> >
> >  0x3cb08 [0x28]: PERF_RECORD_COUNTER_READ:
> uncore_imc_1/cas_count_read/C
> >                 CPU 18: val 1315 ena 2001918 run 2001918
> >
> >  0x3dba0 [0x28]: PERF_RECORD_COUNTER_READ:
> uncore_imc_1/cas_count_read/C
> >                 CPU 0: val 1588 ena 12191520 run 12191520
> >
> >  0x3dbc8 [0x28]: PERF_RECORD_COUNTER_READ:
> uncore_imc_1/cas_count_read/C
> >                 CPU 18: val 1686 ena 12162202 run 12162202
> >
> >  $perf report --stdio --socket-filter 0
> >
> >  # To display the perf.data header info, please use --header/--header-
> only options.
> >  #
> >  #
> >  # Total Lost Samples: 0
> >  #
> >  # Samples: 17K of event 'cycles'
> >  # Event count (approx.): 10119095556
> >  # Processor Socket: 0
> >  #
> >  # Overhead  Command       Shared Object        Symbol
> >  # ........  ............  ...................  ..................................
> >  #
> >     97.68%  tchain_edit   tchain_edit          [.] f3
> >      0.07%  tchain_edit   tchain_edit          [.] f2
> >      0.04%  swapper       [kernel.vmlinux]     [k] run_timer_softirq
> >      0.04%  swapper       [kernel.vmlinux]     [k] find_busiest_group
> >
> >  # Samples: 0  of event 'uncore_imc_1/cas_count_read/C'
> >  # Event count (approx.): 0
> >  # Processor Socket: 0
> >  # uncore_imc_1/cas_count_read/C: 35937
> 
> I think we'll need special output/display for non sampling events,
> something like extra window in TUI and distinguished output in stdio, the
> above is hacked sampling output ;-)

I think it depends on what way we finally use.

If we use the way which merging perf.data from perf stat and record, I think
we need special output for the data from perf stat in TUI/stdio.

But if we use the way counter read (:C), I think the best place to show the
counter read results is the header/title (just like the patch did). Because the
results are the aggregate counts during the whole sampling process.
Something like,
# Event count: 35937 of event 'uncore_imc_1/cas_count_read/C'

Thanks,
Kan

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1232473

On Thu, Sep 24, 2015 at 07:47:40PM +0000, Liang, Kan wrote:
> 
> 
> > On Tue, Sep 22, 2015 at 10:13:33AM -0400, kan.liang@intel.com wrote:
> > > From: Kan Liang <kan.liang@intel.com>
> > >
> > > The patch series intends to read counter statistics with fixed
> > > frequency during sampling. The instant benefit is that we can read
> > > memory bandwidth from uncore event during cpu PMU event is
> > sampling.
> > >
> > > Introduce 'C' event/group modifier. The event with this modifier will
> > > do counting not sampling. If a group with this modifier, only group
> > > leader do sampling. The counter statistics will be wrote in new RECORD
> > > type PERF_RECORD_COUNTER_READ and stored in perf.data.
> > > So perf report can present the counter statistics data accordingly.
> > >
> > > There may be an alternative way to get counter statistics during
> > > sampling by running perf record and perf stat together by script.
> > > But the script way have various issue and complex to parses the
> > > output.
> > 
> > just a thought, but isn't the way then llow to store the data from perf
> > stat? ;-) and be able to merge perf.data-s from perf record and stat
> > afterwards
> 
> Yes, the way to store the data from perf stat is better than pure script
> way. I guess your patch "perf stat record" can do that, right?
> 
> If so, how should we run perf record and stat in parallel? By scripts or
> modify perf record/stat? 
> 
> Also, we need an option in perf report to merge the perf.data-s. Right?

either that or extra step with 'perf data merge' or somthing like that

SNIP

> > but if we go this way I think we should keep/allow all the options perf stat
> 
> Do you mean something like "perf record stat"?
> That's not the way I designed. I don't want to run perf record and perf stat
> together in one command.
> 
> I just want to do similar thing like what sample read did. Sample read can read
> counters on each sample. While the counter read can read counters in a fix

yea, but sample read stores data into ring buffer in the kernel
while you read the data like in perf stat

> frequency (set by --counter-read-interval). So it's an extension of perf record.
> It applies all possible options of perf record, like -C -a -g... 
> I introduce a new option --counter-read-interval is because that there is no
>  interval options in perf record.

the way I see it you implemented 'perf stat' logic within record command
you create counter (non sampling) and read it via read syscall

I think it's good idea, but I think we should follow the way we do
in perf stat command and reuse the interface (and code)

like having the 'stat' keyword separating the non-sampling config:

  $ perf record -e cycles stat -e 'uncore_imc_1/cas_count_read/' -I 10000 - ./tchain_edit

just an idea.. but I dont think the :C modifier is a good way

SNIP

> > I think we'll need special output/display for non sampling events,
> > something like extra window in TUI and distinguished output in stdio, the
> > above is hacked sampling output ;-)
> 
> I think it depends on what way we finally use.
> 
> If we use the way which merging perf.data from perf stat and record, I think
> we need special output for the data from perf stat in TUI/stdio.
> 
> But if we use the way counter read (:C), I think the best place to show the
> counter read results is the header/title (just like the patch did). Because the
> results are the aggregate counts during the whole sampling process.
> Something like,
> # Event count: 35937 of event 'uncore_imc_1/cas_count_read/C'

hum, how the --counter-read-interval data displayed then? it's not single number right?


jirka
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1232863

> On Thu, Sep 24, 2015 at 07:47:40PM +0000, Liang, Kan wrote:
> >
> >
> > > On Tue, Sep 22, 2015 at 10:13:33AM -0400, kan.liang@intel.com wrote:
> > > > From: Kan Liang <kan.liang@intel.com>
> > > >
> > > > The patch series intends to read counter statistics with fixed
> > > > frequency during sampling. The instant benefit is that we can read
> > > > memory bandwidth from uncore event during cpu PMU event is
> > > sampling.
> > > >
> > > > Introduce 'C' event/group modifier. The event with this modifier
> > > > will do counting not sampling. If a group with this modifier, only
> > > > group leader do sampling. The counter statistics will be wrote in
> > > > new RECORD type PERF_RECORD_COUNTER_READ and stored in
> perf.data.
> > > > So perf report can present the counter statistics data accordingly.
> > > >
> > > > There may be an alternative way to get counter statistics during
> > > > sampling by running perf record and perf stat together by script.
> > > > But the script way have various issue and complex to parses the
> > > > output.
> > >
> > > just a thought, but isn't the way then llow to store the data from
> > > perf stat? ;-) and be able to merge perf.data-s from perf record and
> > > stat afterwards
> >
> > Yes, the way to store the data from perf stat is better than pure
> > script way. I guess your patch "perf stat record" can do that, right?
> >
> > If so, how should we run perf record and stat in parallel? By scripts
> > or modify perf record/stat?
> >
> > Also, we need an option in perf report to merge the perf.data-s. Right?
> 
> either that or extra step with 'perf data merge' or somthing like that
>

Any update about "perf stat record" patch set? That will help a lot, if
we finally choose the 'perf data merge' way. Right?  
 
> SNIP
> 
> > > but if we go this way I think we should keep/allow all the options
> > > perf stat
> >
> > Do you mean something like "perf record stat"?
> > That's not the way I designed. I don't want to run perf record and
> > perf stat together in one command.
> >
> > I just want to do similar thing like what sample read did. Sample read
> > can read counters on each sample. While the counter read can read
> > counters in a fix
> 
> yea, but sample read stores data into ring buffer in the kernel while you
> read the data like in perf stat
> 
> > frequency (set by --counter-read-interval). So it's an extension of perf
> record.
> > It applies all possible options of perf record, like -C -a -g...
> > I introduce a new option --counter-read-interval is because that there
> > is no  interval options in perf record.
> 
> the way I see it you implemented 'perf stat' logic within record command
> you create counter (non sampling) and read it via read syscall
> 
> I think it's good idea, but I think we should follow the way we do in perf
> stat command and reuse the interface (and code)
> 
> like having the 'stat' keyword separating the non-sampling config:
> 
>   $ perf record -e cycles stat -e 'uncore_imc_1/cas_count_read/' -I 10000 -
>  ./tchain_edit
>

Another thing is that there is limitation for --interval-print in perf stat.
The interval must >= 100ms. However, we need the interval >=10ms.

Any idea about where 100ms is from? Print limit?  
 
If we choose this way, I think we need to introduce a new option for perf
stat to break up the limitation. 

> just an idea.. but I dont think the :C modifier is a good way
>
> SNIP
> 
> > > I think we'll need special output/display for non sampling events,
> > > something like extra window in TUI and distinguished output in
> > > stdio, the above is hacked sampling output ;-)
> >
> > I think it depends on what way we finally use.
> >
> > If we use the way which merging perf.data from perf stat and record, I
> > think we need special output for the data from perf stat in TUI/stdio.
> >
> > But if we use the way counter read (:C), I think the best place to
> > show the counter read results is the header/title (just like the patch
> > did). Because the results are the aggregate counts during the whole
> sampling process.
> > Something like,
> > # Event count: 35937 of event 'uncore_imc_1/cas_count_read/C'
> 
> hum, how the --counter-read-interval data displayed then? it's not single
> number right?
>
No matter which way we choose, I think the output should be similar.

As my original design, perf only output every --counter-read-interval data
in perf report -D.
For tui and stdio, it only output the aggregate number. So, yes, single number.

I think it should be enough. In tui/stdio, perf gives user a roughly image by
the total number during the whole sampling process. If they want details,
they can check by report -D.
Considering the interval is only 10ms, if perf output everything in tui/stdio,
the output is too huge.

Thanks,
Kan
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1233741

On Fri, Sep 25, 2015 at 02:57:14PM +0000, Liang, Kan wrote:

SNIP

> > >
> > > Yes, the way to store the data from perf stat is better than pure
> > > script way. I guess your patch "perf stat record" can do that, right?
> > >
> > > If so, how should we run perf record and stat in parallel? By scripts
> > > or modify perf record/stat?
> > >
> > > Also, we need an option in perf report to merge the perf.data-s. Right?
> > 
> > either that or extra step with 'perf data merge' or somthing like that
> >
> 
> Any update about "perf stat record" patch set? That will help a lot, if

I'll try to post new version this week

> we finally choose the 'perf data merge' way. Right?  

I think we could do both ways.. let user choose whatever is more convenient

SNIP

> > 
> > the way I see it you implemented 'perf stat' logic within record command
> > you create counter (non sampling) and read it via read syscall
> > 
> > I think it's good idea, but I think we should follow the way we do in perf
> > stat command and reuse the interface (and code)
> > 
> > like having the 'stat' keyword separating the non-sampling config:
> > 
> >   $ perf record -e cycles stat -e 'uncore_imc_1/cas_count_read/' -I 10000 -
> >  ./tchain_edit
> >
> 
> Another thing is that there is limitation for --interval-print in perf stat.
> The interval must >= 100ms. However, we need the interval >=10ms.
> 
> Any idea about where 100ms is from? Print limit?  

I don't recall any reason for this limitation, IMO it was just convenient
to have higher unit because lower wasn't needed.. so I think we can change
it do 10ms

SNIP

> > 
> > hum, how the --counter-read-interval data displayed then? it's not single
> > number right?
> >
> No matter which way we choose, I think the output should be similar.
> 
> As my original design, perf only output every --counter-read-interval data
> in perf report -D.
> For tui and stdio, it only output the aggregate number. So, yes, single number.
> 
> I think it should be enough. In tui/stdio, perf gives user a roughly image by
> the total number during the whole sampling process. If they want details,
> they can check by report -D.
> Considering the interval is only 10ms, if perf output everything in tui/stdio,
> the output is too huge.

what is the reason to read the counter multiple times if you display only
single number at the end? overflow issues?

thanks,
jirka
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1234246

> > >
> > > hum, how the --counter-read-interval data displayed then? it's not
> > > single number right?
> > >
> > No matter which way we choose, I think the output should be similar.
> >
> > As my original design, perf only output every --counter-read-interval
> > data in perf report -D.
> > For tui and stdio, it only output the aggregate number. So, yes, single
> number.
> >
> > I think it should be enough. In tui/stdio, perf gives user a roughly
> > image by the total number during the whole sampling process. If they
> > want details, they can check by report -D.
> > Considering the interval is only 10ms, if perf output everything in
> > tui/stdio, the output is too huge.
> 
> what is the reason to read the counter multiple times

For doing sophisticated memory bandwidth analysis, it requires 10-20ms
interval to read uncore counter.

> if you display only
> single number at the end? overflow issues?
>

Display issue. If we set 10ms interval and do perf record stat for 10s, there
will be 1000 records per cpu. For a system with 64 cpu, there will be 64,000
number. We cannot show all of them in tui/stdio mode.
I think it's better to only show the total number in tui/stdio mode.
If the user want to do sophisticated analysis, they can use -D to dump all
records.

Thanks,
Kan 
 
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web