Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1405797 > unrolled thread

[PATCH v4 0/7] perf tools: Support overwritable ring buffer

Started byWang Nan <wangnan0@huawei.com>
First post2016-05-24 04:40 +0200
Last post2016-05-25 04:30 +0200
Articles 6 — 3 participants

Back to article view | Back to linux.kernel

Contents

  [PATCH v4 0/7] perf tools: Support overwritable ring buffer Wang Nan <wangnan0@huawei.com> - 2016-05-24 04:40 +0200
    [PATCH v4 2/7] perf tools: Don't poll and mmap overwritable events Wang Nan <wangnan0@huawei.com> - 2016-05-24 04:40 +0200
    [PATCH v4 3/7] perf tools: Enable overwrite settings Wang Nan <wangnan0@huawei.com> - 2016-05-24 04:40 +0200
      Re: [PATCH v4 3/7] perf tools: Enable overwrite settings Arnaldo Carvalho de Melo <acme@kernel.org> - 2016-05-24 20:50 +0200
      Re: [PATCH v4 3/7] perf tools: Enable overwrite settings Arnaldo Carvalho de Melo <acme@kernel.org> - 2016-05-24 20:50 +0200
        Re: [PATCH v4 3/7] perf tools: Enable overwrite settings "Wangnan (F)" <wangnan0@huawei.com> - 2016-05-25 04:30 +0200

#1405797 — [PATCH v4 0/7] perf tools: Support overwritable ring buffer

FromWang Nan <wangnan0@huawei.com>
Date2016-05-24 04:40 +0200
Subject[PATCH v4 0/7] perf tools: Support overwritable ring buffer
Message-ID<rCaqd-5CJ-3@gated-at.bofh.it>
This patch set enables daemonized perf recording by utilizing
overwritable backward ring buffer. With this feature one can
put perf background, and dump ring buffer records by a SIGUSR2
when he/she find something unusual. For example, following
command record system calls, schedule events and samples on cpu cycles
continously:

 # perf record -g -e cycles -e raw_syscalls:*/call-graph=no/ \
                  -e sched:sched_switch/call-graph=no/ \
                  --switch-output --overwrite -a

Then by sending SIGUSR2 to perf when lagging is happen, we get multiple
perf.data output, each of them correspond a abnormal event, and the data
size is reasonable:

 # ls -l ./perf.data*
 -rw------- 1 root root 5122165 May 13 23:51 ./perf.data.2016051323511683
 -rw------- 1 root root 5135093 May 13 23:51 ./perf.data.2016051323512107
 -rw------- 1 root root 5135213 May 13 23:51 ./perf.data.2016051323512215
 -rw------- 1 root root 5135157 May 13 23:51 ./perf.data.2016051323512387

v1 -> v2: Totally redesign: drop the principle of 'channal', use
          auxiliary evlist instead. Fix missing documentation.

v2 -> v3: Rename perf_evlist__toggle_paused() to perf_evlist__pause/resume.

v3 -> v4: Update commit message to describe auxiliary evlist more clearly.

Wang Nan (7):
  perf evlist: Introduce aux perf evlist
  perf tools: Don't poll and mmap overwritable events
  perf tools: Enable overwrite settings
  perf record: Introduce rec->overwrite_evlist for overwritable events
  perf record: Toggle overwrite ring buffer for reading
  perf tools: Don't warn about out of order event if write_backward is
    used
  perf tools: Check write_backward during evlist config

 tools/perf/Documentation/perf-record.txt |  14 ++
 tools/perf/arch/x86/util/tsc.c           |   2 +
 tools/perf/builtin-record.c              | 283 +++++++++++++++++++++++++++----
 tools/perf/perf.h                        |   1 +
 tools/perf/util/evlist.c                 |  52 ++++--
 tools/perf/util/evlist.h                 |   3 +
 tools/perf/util/evsel.c                  |  27 +--
 tools/perf/util/evsel.h                  |  15 ++
 tools/perf/util/parse-events.c           |  20 ++-
 tools/perf/util/parse-events.h           |   2 +
 tools/perf/util/parse-events.l           |   2 +
 tools/perf/util/record.c                 |  17 ++
 tools/perf/util/session.c                |  22 ++-
 13 files changed, 399 insertions(+), 61 deletions(-)

Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Wang Nan <wangnan0@huawei.com>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com

-- 
1.8.3.4

[toc] | [next] | [standalone]

#1405798 — [PATCH v4 2/7] perf tools: Don't poll and mmap overwritable events

FromWang Nan <wangnan0@huawei.com>
Date2016-05-24 04:40 +0200
Subject[PATCH v4 2/7] perf tools: Don't poll and mmap overwritable events
Message-ID<rCaqd-5CJ-17@gated-at.bofh.it>
In reply to#1405797
There's no need to receive events from overwritable ring buffer. Instead,
perf should make them run background until something happen. This patch
makes normal events from overwrite events ignored.

Overwritable events must be mapped readonly and backward, so if evlist
and evsel is not match (evsel->overwrite is true but either evlist is
read/write or evlist is not backward, and vice versa), skip mapping it.

It is possible that all events in an evlist are overwritable.
perf_event__synth_time_conv() should not crash in this case. record__pick_pc()
is used to check avaliability. Further commits will expand it when we
introduce auxiliary evlists and have multiple mmaps.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Signed-off-by: He Kuang <hekuang@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
---
 tools/perf/arch/x86/util/tsc.c |  2 ++
 tools/perf/builtin-record.c    |  9 ++++++++-
 tools/perf/util/evlist.c       | 23 +++++++++++++++++++----
 3 files changed, 29 insertions(+), 5 deletions(-)

diff --git a/tools/perf/arch/x86/util/tsc.c b/tools/perf/arch/x86/util/tsc.c
index 357f1b1..2e5567c 100644
--- a/tools/perf/arch/x86/util/tsc.c
+++ b/tools/perf/arch/x86/util/tsc.c
@@ -62,6 +62,8 @@ int perf_event__synth_time_conv(const struct perf_event_mmap_page *pc,
 	struct perf_tsc_conversion tc;
 	int err;
 
+	if (!pc)
+		return 0;
 	err = perf_read_tsc_conversion(pc, &tc);
 	if (err == -EOPNOTSUPP)
 		return 0;
diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index dc3fcb5..d4cf1b0 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -655,6 +655,13 @@ perf_event__synth_time_conv(const struct perf_event_mmap_page *pc __maybe_unused
 	return 0;
 }
 
+static const struct perf_event_mmap_page *record__pick_pc(struct record *rec)
+{
+	if (rec->evlist && rec->evlist->mmap && rec->evlist->mmap[0].base)
+		return rec->evlist->mmap[0].base;
+	return NULL;
+}
+
 static int record__synthesize(struct record *rec)
 {
 	struct perf_session *session = rec->session;
@@ -692,7 +699,7 @@ static int record__synthesize(struct record *rec)
 		}
 	}
 
-	err = perf_event__synth_time_conv(rec->evlist->mmap[0].base, tool,
+	err = perf_event__synth_time_conv(record__pick_pc(rec), tool,
 					  process_synthesized_event, machine);
 	if (err)
 		goto out;
diff --git a/tools/perf/util/evlist.c b/tools/perf/util/evlist.c
index 55cea69..fbd0d47 100644
--- a/tools/perf/util/evlist.c
+++ b/tools/perf/util/evlist.c
@@ -463,9 +463,9 @@ int perf_evlist__alloc_pollfd(struct perf_evlist *evlist)
 	return 0;
 }
 
-static int __perf_evlist__add_pollfd(struct perf_evlist *evlist, int fd, int idx)
+static int __perf_evlist__add_pollfd(struct perf_evlist *evlist, int fd, int idx, short revent)
 {
-	int pos = fdarray__add(&evlist->pollfd, fd, POLLIN | POLLERR | POLLHUP);
+	int pos = fdarray__add(&evlist->pollfd, fd, revent | POLLERR | POLLHUP);
 	/*
 	 * Save the idx so that when we filter out fds POLLHUP'ed we can
 	 * close the associated evlist->mmap[] entry.
@@ -481,7 +481,7 @@ static int __perf_evlist__add_pollfd(struct perf_evlist *evlist, int fd, int idx
 
 int perf_evlist__add_pollfd(struct perf_evlist *evlist, int fd)
 {
-	return __perf_evlist__add_pollfd(evlist, fd, -1);
+	return __perf_evlist__add_pollfd(evlist, fd, -1, POLLIN);
 }
 
 static void perf_evlist__munmap_filtered(struct fdarray *fda, int fd)
@@ -984,15 +984,28 @@ static int __perf_evlist__mmap(struct perf_evlist *evlist, int idx,
 	return 0;
 }
 
+static bool
+perf_evlist__should_poll(struct perf_evlist *evlist __maybe_unused,
+			 struct perf_evsel *evsel)
+{
+	if (evsel->overwrite)
+		return false;
+	return true;
+}
+
 static int perf_evlist__mmap_per_evsel(struct perf_evlist *evlist, int idx,
 				       struct mmap_params *mp, int cpu,
 				       int thread, int *output)
 {
 	struct perf_evsel *evsel;
+	int revent;
 
 	evlist__for_each(evlist->parent, evsel) {
 		int fd;
 
+		if (evsel->overwrite != (evlist->overwrite && evlist->backward))
+			continue;
+
 		if (evsel->system_wide && thread)
 			continue;
 
@@ -1009,6 +1022,8 @@ static int perf_evlist__mmap_per_evsel(struct perf_evlist *evlist, int idx,
 			perf_evlist__mmap_get(evlist, idx);
 		}
 
+		revent = perf_evlist__should_poll(evlist, evsel) ? POLLIN : 0;
+
 		/*
 		 * The system_wide flag causes a selected event to be opened
 		 * always without a pid.  Consequently it will never get a
@@ -1017,7 +1032,7 @@ static int perf_evlist__mmap_per_evsel(struct perf_evlist *evlist, int idx,
 		 * Therefore don't add it for polling.
 		 */
 		if (!evsel->system_wide &&
-		    __perf_evlist__add_pollfd(evlist->parent, fd, idx) < 0) {
+		    __perf_evlist__add_pollfd(evlist->parent, fd, idx, revent) < 0) {
 			perf_evlist__mmap_put(evlist, idx);
 			return -1;
 		}
-- 
1.8.3.4

[toc] | [prev] | [next] | [standalone]

#1405799 — [PATCH v4 3/7] perf tools: Enable overwrite settings

FromWang Nan <wangnan0@huawei.com>
Date2016-05-24 04:40 +0200
Subject[PATCH v4 3/7] perf tools: Enable overwrite settings
Message-ID<rCaqe-5CJ-19@gated-at.bofh.it>
In reply to#1405797
This patch allows following config terms and option:

Globally setting events to overwrite;

 # perf record --overwrite ...

Set specific events to be overwrite or no-overwrite.

 # perf record --event cycles/overwrite/ ...
 # perf record --event cycles/no-overwrite/ ...

Add missing config terms and update config term array size because the
longest string length is changed.

For overwritable events, automatically select attr.write_backward since
perf requires it to be backward for reading.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Signed-off-by: He Kuang <hekuang@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
---
 tools/perf/Documentation/perf-record.txt | 14 ++++++++++++++
 tools/perf/builtin-record.c              |  1 +
 tools/perf/perf.h                        |  1 +
 tools/perf/util/evsel.c                  | 12 ++++++++++++
 tools/perf/util/evsel.h                  |  2 ++
 tools/perf/util/parse-events.c           | 20 ++++++++++++++++++--
 tools/perf/util/parse-events.h           |  2 ++
 tools/perf/util/parse-events.l           |  2 ++
 8 files changed, 52 insertions(+), 2 deletions(-)

diff --git a/tools/perf/Documentation/perf-record.txt b/tools/perf/Documentation/perf-record.txt
index 8dbee83..f5cb932 100644
--- a/tools/perf/Documentation/perf-record.txt
+++ b/tools/perf/Documentation/perf-record.txt
@@ -360,6 +360,20 @@ particular perf.data snapshot should be kept or not.
 
 Implies --timestamp-filename, --no-buildid and --no-buildid-cache.
 
+--overwrite::
+Makes all events use overwritable ring buffer. Event with overwritable ring
+buffer works like a flight recorder: when buffer gets full, instead of dumping
+records into output file, kernel overwrites old records silently. Perf dumps
+data from overwritable ring buffer when switching output (see --switch-output)
+and before terminate.
+
+Perf behaves like a daemon when '--overwrite' and '--switch-output' are
+provided. It record and drop events in background, and dumps data when
+something unusual is detected.
+
+'overwrite' attribute can also be set or canceled for specific event using
+config terms like 'cycles/overwrite/' and 'instructions/no-overwrite/'.
+
 SEE ALSO
 --------
 linkperf:perf-stat[1], linkperf:perf-list[1]
diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index d4cf1b0..9611380 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -1310,6 +1310,7 @@ struct option __record_options[] = {
 	OPT_BOOLEAN_SET('i', "no-inherit", &record.opts.no_inherit,
 			&record.opts.no_inherit_set,
 			"child tasks do not inherit counters"),
+	OPT_BOOLEAN(0, "overwrite", &record.opts.overwrite, "use overwrite mode"),
 	OPT_UINTEGER('F', "freq", &record.opts.user_freq, "profile at this frequency"),
 	OPT_CALLBACK('m', "mmap-pages", &record.opts, "pages[,pages]",
 		     "number of mmap data pages and AUX area tracing mmap pages",
diff --git a/tools/perf/perf.h b/tools/perf/perf.h
index cd8f1b1..608b42b 100644
--- a/tools/perf/perf.h
+++ b/tools/perf/perf.h
@@ -59,6 +59,7 @@ struct record_opts {
 	bool	     record_switch_events;
 	bool	     all_kernel;
 	bool	     all_user;
+	bool	     overwrite;
 	unsigned int freq;
 	unsigned int mmap_pages;
 	unsigned int auxtrace_mmap_pages;
diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
index 02c177d..6330a4f 100644
--- a/tools/perf/util/evsel.c
+++ b/tools/perf/util/evsel.c
@@ -671,11 +671,22 @@ static void apply_config_terms(struct perf_evsel *evsel,
 			 */
 			attr->inherit = term->val.inherit ? 1 : 0;
 			break;
+		case PERF_EVSEL__CONFIG_TERM_OVERWRITE:
+			evsel->overwrite = term->val.overwrite ? 1 : 0;
+			break;
 		default:
 			break;
 		}
 	}
 
+	/*
+	 * Set backward after config term processing because it is
+	 * possible to set overwrite globally, without config
+	 * terms.
+	 */
+	if (evsel->overwrite)
+		attr->write_backward = 1;
+
 	/* User explicitly set per-event callgraph, clear the old setting and reset. */
 	if ((callgraph_buf != NULL) || (dump_size > 0)) {
 
@@ -747,6 +758,7 @@ void perf_evsel__config(struct perf_evsel *evsel, struct record_opts *opts,
 
 	attr->sample_id_all = perf_missing_features.sample_id_all ? 0 : 1;
 	attr->inherit	    = !opts->no_inherit;
+	evsel->overwrite    = opts->overwrite;
 
 	perf_evsel__set_sample_bit(evsel, IP);
 	perf_evsel__set_sample_bit(evsel, TID);
diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h
index c1f1015..bce99fa 100644
--- a/tools/perf/util/evsel.h
+++ b/tools/perf/util/evsel.h
@@ -44,6 +44,7 @@ enum {
 	PERF_EVSEL__CONFIG_TERM_CALLGRAPH,
 	PERF_EVSEL__CONFIG_TERM_STACK_USER,
 	PERF_EVSEL__CONFIG_TERM_INHERIT,
+	PERF_EVSEL__CONFIG_TERM_OVERWRITE,
 	PERF_EVSEL__CONFIG_TERM_MAX,
 };
 
@@ -57,6 +58,7 @@ struct perf_evsel_config_term {
 		char	*callgraph;
 		u64	stack_user;
 		bool	inherit;
+		bool	overwrite;
 	} val;
 };
 
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index bcbc983..85f813d 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -900,6 +900,8 @@ static const char *config_term_names[__PARSE_EVENTS__TERM_TYPE_NR] = {
 	[PARSE_EVENTS__TERM_TYPE_STACKSIZE]		= "stack-size",
 	[PARSE_EVENTS__TERM_TYPE_NOINHERIT]		= "no-inherit",
 	[PARSE_EVENTS__TERM_TYPE_INHERIT]		= "inherit",
+	[PARSE_EVENTS__TERM_TYPE_OVERWRITE]		= "overwrite",
+	[PARSE_EVENTS__TERM_TYPE_NOOVERWRITE]		= "no-overwrite",
 };
 
 static bool config_term_shrinked;
@@ -992,6 +994,12 @@ do {									   \
 	case PARSE_EVENTS__TERM_TYPE_NOINHERIT:
 		CHECK_TYPE_VAL(NUM);
 		break;
+	case PARSE_EVENTS__TERM_TYPE_OVERWRITE:
+		CHECK_TYPE_VAL(NUM);
+		break;
+	case PARSE_EVENTS__TERM_TYPE_NOOVERWRITE:
+		CHECK_TYPE_VAL(NUM);
+		break;
 	case PARSE_EVENTS__TERM_TYPE_NAME:
 		CHECK_TYPE_VAL(STR);
 		break;
@@ -1040,6 +1048,8 @@ static int config_term_tracepoint(struct perf_event_attr *attr,
 	case PARSE_EVENTS__TERM_TYPE_STACKSIZE:
 	case PARSE_EVENTS__TERM_TYPE_INHERIT:
 	case PARSE_EVENTS__TERM_TYPE_NOINHERIT:
+	case PARSE_EVENTS__TERM_TYPE_OVERWRITE:
+	case PARSE_EVENTS__TERM_TYPE_NOOVERWRITE:
 		return config_term_common(attr, term, err);
 	default:
 		if (err) {
@@ -1109,6 +1119,12 @@ do {								\
 		case PARSE_EVENTS__TERM_TYPE_NOINHERIT:
 			ADD_CONFIG_TERM(INHERIT, inherit, term->val.num ? 0 : 1);
 			break;
+		case PARSE_EVENTS__TERM_TYPE_OVERWRITE:
+			ADD_CONFIG_TERM(OVERWRITE, overwrite, term->val.num ? 1 : 0);
+			break;
+		case PARSE_EVENTS__TERM_TYPE_NOOVERWRITE:
+			ADD_CONFIG_TERM(OVERWRITE, overwrite, term->val.num ? 0 : 1);
+			break;
 		default:
 			break;
 		}
@@ -2322,9 +2338,9 @@ static void config_terms_list(char *buf, size_t buf_sz)
 char *parse_events_formats_error_string(char *additional_terms)
 {
 	char *str;
-	/* "branch_type" is the longest name */
+	/* "no-overwrite" is the longest name */
 	char static_terms[__PARSE_EVENTS__TERM_TYPE_NR *
-			  (sizeof("branch_type") - 1)];
+			  (sizeof("no-overwrite") - 1)];
 
 	config_terms_list(static_terms, sizeof(static_terms));
 	/* valid terms */
diff --git a/tools/perf/util/parse-events.h b/tools/perf/util/parse-events.h
index d740c3c..f341d9d 100644
--- a/tools/perf/util/parse-events.h
+++ b/tools/perf/util/parse-events.h
@@ -68,6 +68,8 @@ enum {
 	PARSE_EVENTS__TERM_TYPE_STACKSIZE,
 	PARSE_EVENTS__TERM_TYPE_NOINHERIT,
 	PARSE_EVENTS__TERM_TYPE_INHERIT,
+	PARSE_EVENTS__TERM_TYPE_NOOVERWRITE,
+	PARSE_EVENTS__TERM_TYPE_OVERWRITE,
 	__PARSE_EVENTS__TERM_TYPE_NR,
 };
 
diff --git a/tools/perf/util/parse-events.l b/tools/perf/util/parse-events.l
index 1477fbc..cc4c426 100644
--- a/tools/perf/util/parse-events.l
+++ b/tools/perf/util/parse-events.l
@@ -201,6 +201,8 @@ call-graph		{ return term(yyscanner, PARSE_EVENTS__TERM_TYPE_CALLGRAPH); }
 stack-size		{ return term(yyscanner, PARSE_EVENTS__TERM_TYPE_STACKSIZE); }
 inherit			{ return term(yyscanner, PARSE_EVENTS__TERM_TYPE_INHERIT); }
 no-inherit		{ return term(yyscanner, PARSE_EVENTS__TERM_TYPE_NOINHERIT); }
+overwrite		{ return term(yyscanner, PARSE_EVENTS__TERM_TYPE_OVERWRITE); }
+no-overwrite		{ return term(yyscanner, PARSE_EVENTS__TERM_TYPE_NOOVERWRITE); }
 ,			{ return ','; }
 "/"			{ BEGIN(INITIAL); return '/'; }
 {name_minus}		{ return str(yyscanner, PE_NAME); }
-- 
1.8.3.4

[toc] | [prev] | [next] | [standalone]

#1406387 — Re: [PATCH v4 3/7] perf tools: Enable overwrite settings

FromArnaldo Carvalho de Melo <acme@kernel.org>
Date2016-05-24 20:50 +0200
SubjectRe: [PATCH v4 3/7] perf tools: Enable overwrite settings
Message-ID<rCpyV-6Uj-1@gated-at.bofh.it>
In reply to#1405799
Em Tue, May 24, 2016 at 03:40:30PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Tue, May 24, 2016 at 02:29:00AM +0000, Wang Nan escreveu:
> > This patch allows following config terms and option:
> > 
> > Globally setting events to overwrite;
> > 
> >  # perf record --overwrite ...
> > 
> > Set specific events to be overwrite or no-overwrite.
> > 
> >  # perf record --event cycles/overwrite/ ...
> >  # perf record --event cycles/no-overwrite/ ...
> 
> So, based on this chunk of documentation in this patch:

BTW, I applied the first 2 patches.

- Arnaldo

[toc] | [prev] | [next] | [standalone]

#1406389 — Re: [PATCH v4 3/7] perf tools: Enable overwrite settings

FromArnaldo Carvalho de Melo <acme@kernel.org>
Date2016-05-24 20:50 +0200
SubjectRe: [PATCH v4 3/7] perf tools: Enable overwrite settings
Message-ID<rCpyV-6Uj-3@gated-at.bofh.it>
In reply to#1405799
Em Tue, May 24, 2016 at 02:29:00AM +0000, Wang Nan escreveu:
> This patch allows following config terms and option:
> 
> Globally setting events to overwrite;
> 
>  # perf record --overwrite ...
> 
> Set specific events to be overwrite or no-overwrite.
> 
>  # perf record --event cycles/overwrite/ ...
>  # perf record --event cycles/no-overwrite/ ...

So, based on this chunk of documentation in this patch:

<quote>
Perf dumps data from overwritable ring buffer when switching output (see
--switch-output) and before terminate.
</>

I tried:

No --overwrite:

  # perf record -e syscalls:*enter_nanosleep* usleep 1
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.019 MB perf.data (1 samples) ]
  # perf evlist -v
  syscalls:sys_enter_nanosleep: type: 2, size: 112, config: 0x132, { sample_period, sample_freq }: 1, sample_type: IP|TID|TIME|CPU|PERIOD|RAW, disabled: 1, inherit: 1, mmap: 1, comm: 1, enable_on_exec: 1, task: 1, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1
  # Tip: use 'perf evlist --trace-fields' to show fields for tracepoint events
  # perf script
          usleep 29416 [002] 220099.782982: syscalls:sys_enter_nanosleep: rqtp: 0x7ffc21f73cc0, rmtp: 0x00000000

Now I went on to try this new --overwrite thing:

  # perf record --overwrite -e syscalls:*enter_nanosleep* usleep 1
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.019 MB perf.data ]
  # perf evlist -v
syscalls:sys_enter_nanosleep: type: 2, size: 112, config: 0x132, { sample_period, sample_freq }: 1, sample_type: IP|TID|TIME|CPU|PERIOD|RAW, disabled: 1, inherit: 1, mmap: 1, comm: 1, enable_on_exec: 1, task: 1, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1, write_backward: 1
  # Tip: use 'perf evlist --trace-fields' to show fields for tracepoint events
  # perf script
  # 

So it hasn't recorded anything at anytime, i.e. I expected, based on the
documentation provided, that it would get what was in its buffer, to be written, 
i.e. the single "syscalls:sys_enter_nanosleep" event that took place in that
workload.

So I'm now trying it together with --switch-output, but I just get one
timestamp suffixed perf.data file, empty, without that event that I know took
place.

Care to ellaborate here?

- Arnaldo
 
> Add missing config terms and update config term array size because the
> longest string length is changed.
> 
> For overwritable events, automatically select attr.write_backward since
> perf requires it to be backward for reading.
> 
> Signed-off-by: Wang Nan <wangnan0@huawei.com>
> Signed-off-by: He Kuang <hekuang@huawei.com>
> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
> Cc: Jiri Olsa <jolsa@kernel.org>
> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
> Cc: Namhyung Kim <namhyung@kernel.org>
> Cc: Zefan Li <lizefan@huawei.com>
> Cc: pi3orama@163.com
> ---
>  tools/perf/Documentation/perf-record.txt | 14 ++++++++++++++
>  tools/perf/builtin-record.c              |  1 +
>  tools/perf/perf.h                        |  1 +
>  tools/perf/util/evsel.c                  | 12 ++++++++++++
>  tools/perf/util/evsel.h                  |  2 ++
>  tools/perf/util/parse-events.c           | 20 ++++++++++++++++++--
>  tools/perf/util/parse-events.h           |  2 ++
>  tools/perf/util/parse-events.l           |  2 ++
>  8 files changed, 52 insertions(+), 2 deletions(-)
> 
> diff --git a/tools/perf/Documentation/perf-record.txt b/tools/perf/Documentation/perf-record.txt
> index 8dbee83..f5cb932 100644
> --- a/tools/perf/Documentation/perf-record.txt
> +++ b/tools/perf/Documentation/perf-record.txt
> @@ -360,6 +360,20 @@ particular perf.data snapshot should be kept or not.
>  
>  Implies --timestamp-filename, --no-buildid and --no-buildid-cache.
>  
> +--overwrite::
> +Makes all events use overwritable ring buffer. Event with overwritable ring
> +buffer works like a flight recorder: when buffer gets full, instead of dumping
> +records into output file, kernel overwrites old records silently. Perf dumps
> +data from overwritable ring buffer when switching output (see --switch-output)
> +and before terminate.
> +
> +Perf behaves like a daemon when '--overwrite' and '--switch-output' are
> +provided. It record and drop events in background, and dumps data when
> +something unusual is detected.
> +
> +'overwrite' attribute can also be set or canceled for specific event using
> +config terms like 'cycles/overwrite/' and 'instructions/no-overwrite/'.
> +
>  SEE ALSO
>  --------
>  linkperf:perf-stat[1], linkperf:perf-list[1]
> diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
> index d4cf1b0..9611380 100644
> --- a/tools/perf/builtin-record.c
> +++ b/tools/perf/builtin-record.c
> @@ -1310,6 +1310,7 @@ struct option __record_options[] = {
>  	OPT_BOOLEAN_SET('i', "no-inherit", &record.opts.no_inherit,
>  			&record.opts.no_inherit_set,
>  			"child tasks do not inherit counters"),
> +	OPT_BOOLEAN(0, "overwrite", &record.opts.overwrite, "use overwrite mode"),
>  	OPT_UINTEGER('F', "freq", &record.opts.user_freq, "profile at this frequency"),
>  	OPT_CALLBACK('m', "mmap-pages", &record.opts, "pages[,pages]",
>  		     "number of mmap data pages and AUX area tracing mmap pages",
> diff --git a/tools/perf/perf.h b/tools/perf/perf.h
> index cd8f1b1..608b42b 100644
> --- a/tools/perf/perf.h
> +++ b/tools/perf/perf.h
> @@ -59,6 +59,7 @@ struct record_opts {
>  	bool	     record_switch_events;
>  	bool	     all_kernel;
>  	bool	     all_user;
> +	bool	     overwrite;
>  	unsigned int freq;
>  	unsigned int mmap_pages;
>  	unsigned int auxtrace_mmap_pages;
> diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
> index 02c177d..6330a4f 100644
> --- a/tools/perf/util/evsel.c
> +++ b/tools/perf/util/evsel.c
> @@ -671,11 +671,22 @@ static void apply_config_terms(struct perf_evsel *evsel,
>  			 */
>  			attr->inherit = term->val.inherit ? 1 : 0;
>  			break;
> +		case PERF_EVSEL__CONFIG_TERM_OVERWRITE:
> +			evsel->overwrite = term->val.overwrite ? 1 : 0;
> +			break;
>  		default:
>  			break;
>  		}
>  	}
>  
> +	/*
> +	 * Set backward after config term processing because it is
> +	 * possible to set overwrite globally, without config
> +	 * terms.
> +	 */
> +	if (evsel->overwrite)
> +		attr->write_backward = 1;
> +
>  	/* User explicitly set per-event callgraph, clear the old setting and reset. */
>  	if ((callgraph_buf != NULL) || (dump_size > 0)) {
>  
> @@ -747,6 +758,7 @@ void perf_evsel__config(struct perf_evsel *evsel, struct record_opts *opts,
>  
>  	attr->sample_id_all = perf_missing_features.sample_id_all ? 0 : 1;
>  	attr->inherit	    = !opts->no_inherit;
> +	evsel->overwrite    = opts->overwrite;
>  
>  	perf_evsel__set_sample_bit(evsel, IP);
>  	perf_evsel__set_sample_bit(evsel, TID);
> diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h
> index c1f1015..bce99fa 100644
> --- a/tools/perf/util/evsel.h
> +++ b/tools/perf/util/evsel.h
> @@ -44,6 +44,7 @@ enum {
>  	PERF_EVSEL__CONFIG_TERM_CALLGRAPH,
>  	PERF_EVSEL__CONFIG_TERM_STACK_USER,
>  	PERF_EVSEL__CONFIG_TERM_INHERIT,
> +	PERF_EVSEL__CONFIG_TERM_OVERWRITE,
>  	PERF_EVSEL__CONFIG_TERM_MAX,
>  };
>  
> @@ -57,6 +58,7 @@ struct perf_evsel_config_term {
>  		char	*callgraph;
>  		u64	stack_user;
>  		bool	inherit;
> +		bool	overwrite;
>  	} val;
>  };
>  
> diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
> index bcbc983..85f813d 100644
> --- a/tools/perf/util/parse-events.c
> +++ b/tools/perf/util/parse-events.c
> @@ -900,6 +900,8 @@ static const char *config_term_names[__PARSE_EVENTS__TERM_TYPE_NR] = {
>  	[PARSE_EVENTS__TERM_TYPE_STACKSIZE]		= "stack-size",
>  	[PARSE_EVENTS__TERM_TYPE_NOINHERIT]		= "no-inherit",
>  	[PARSE_EVENTS__TERM_TYPE_INHERIT]		= "inherit",
> +	[PARSE_EVENTS__TERM_TYPE_OVERWRITE]		= "overwrite",
> +	[PARSE_EVENTS__TERM_TYPE_NOOVERWRITE]		= "no-overwrite",
>  };
>  
>  static bool config_term_shrinked;
> @@ -992,6 +994,12 @@ do {									   \
>  	case PARSE_EVENTS__TERM_TYPE_NOINHERIT:
>  		CHECK_TYPE_VAL(NUM);
>  		break;
> +	case PARSE_EVENTS__TERM_TYPE_OVERWRITE:
> +		CHECK_TYPE_VAL(NUM);
> +		break;
> +	case PARSE_EVENTS__TERM_TYPE_NOOVERWRITE:
> +		CHECK_TYPE_VAL(NUM);
> +		break;
>  	case PARSE_EVENTS__TERM_TYPE_NAME:
>  		CHECK_TYPE_VAL(STR);
>  		break;
> @@ -1040,6 +1048,8 @@ static int config_term_tracepoint(struct perf_event_attr *attr,
>  	case PARSE_EVENTS__TERM_TYPE_STACKSIZE:
>  	case PARSE_EVENTS__TERM_TYPE_INHERIT:
>  	case PARSE_EVENTS__TERM_TYPE_NOINHERIT:
> +	case PARSE_EVENTS__TERM_TYPE_OVERWRITE:
> +	case PARSE_EVENTS__TERM_TYPE_NOOVERWRITE:
>  		return config_term_common(attr, term, err);
>  	default:
>  		if (err) {
> @@ -1109,6 +1119,12 @@ do {								\
>  		case PARSE_EVENTS__TERM_TYPE_NOINHERIT:
>  			ADD_CONFIG_TERM(INHERIT, inherit, term->val.num ? 0 : 1);
>  			break;
> +		case PARSE_EVENTS__TERM_TYPE_OVERWRITE:
> +			ADD_CONFIG_TERM(OVERWRITE, overwrite, term->val.num ? 1 : 0);
> +			break;
> +		case PARSE_EVENTS__TERM_TYPE_NOOVERWRITE:
> +			ADD_CONFIG_TERM(OVERWRITE, overwrite, term->val.num ? 0 : 1);
> +			break;
>  		default:
>  			break;
>  		}
> @@ -2322,9 +2338,9 @@ static void config_terms_list(char *buf, size_t buf_sz)
>  char *parse_events_formats_error_string(char *additional_terms)
>  {
>  	char *str;
> -	/* "branch_type" is the longest name */
> +	/* "no-overwrite" is the longest name */
>  	char static_terms[__PARSE_EVENTS__TERM_TYPE_NR *
> -			  (sizeof("branch_type") - 1)];
> +			  (sizeof("no-overwrite") - 1)];
>  
>  	config_terms_list(static_terms, sizeof(static_terms));
>  	/* valid terms */
> diff --git a/tools/perf/util/parse-events.h b/tools/perf/util/parse-events.h
> index d740c3c..f341d9d 100644
> --- a/tools/perf/util/parse-events.h
> +++ b/tools/perf/util/parse-events.h
> @@ -68,6 +68,8 @@ enum {
>  	PARSE_EVENTS__TERM_TYPE_STACKSIZE,
>  	PARSE_EVENTS__TERM_TYPE_NOINHERIT,
>  	PARSE_EVENTS__TERM_TYPE_INHERIT,
> +	PARSE_EVENTS__TERM_TYPE_NOOVERWRITE,
> +	PARSE_EVENTS__TERM_TYPE_OVERWRITE,
>  	__PARSE_EVENTS__TERM_TYPE_NR,
>  };
>  
> diff --git a/tools/perf/util/parse-events.l b/tools/perf/util/parse-events.l
> index 1477fbc..cc4c426 100644
> --- a/tools/perf/util/parse-events.l
> +++ b/tools/perf/util/parse-events.l
> @@ -201,6 +201,8 @@ call-graph		{ return term(yyscanner, PARSE_EVENTS__TERM_TYPE_CALLGRAPH); }
>  stack-size		{ return term(yyscanner, PARSE_EVENTS__TERM_TYPE_STACKSIZE); }
>  inherit			{ return term(yyscanner, PARSE_EVENTS__TERM_TYPE_INHERIT); }
>  no-inherit		{ return term(yyscanner, PARSE_EVENTS__TERM_TYPE_NOINHERIT); }
> +overwrite		{ return term(yyscanner, PARSE_EVENTS__TERM_TYPE_OVERWRITE); }
> +no-overwrite		{ return term(yyscanner, PARSE_EVENTS__TERM_TYPE_NOOVERWRITE); }
>  ,			{ return ','; }
>  "/"			{ BEGIN(INITIAL); return '/'; }
>  {name_minus}		{ return str(yyscanner, PE_NAME); }
> -- 
> 1.8.3.4

[toc] | [prev] | [next] | [standalone]

#1406565 — Re: [PATCH v4 3/7] perf tools: Enable overwrite settings

From"Wangnan (F)" <wangnan0@huawei.com>
Date2016-05-25 04:30 +0200
SubjectRe: [PATCH v4 3/7] perf tools: Enable overwrite settings
Message-ID<rCwK5-3qV-7@gated-at.bofh.it>
In reply to#1406389

On 2016/5/25 2:40, Arnaldo Carvalho de Melo wrote:
> Em Tue, May 24, 2016 at 02:29:00AM +0000, Wang Nan escreveu:
>> This patch allows following config terms and option:
>>
>> Globally setting events to overwrite;
>>
>>   # perf record --overwrite ...
>>
>> Set specific events to be overwrite or no-overwrite.
>>
>>   # perf record --event cycles/overwrite/ ...
>>   # perf record --event cycles/no-overwrite/ ...
> So, based on this chunk of documentation in this patch:
>
> <quote>
> Perf dumps data from overwritable ring buffer when switching output (see
> --switch-output) and before terminate.
> </>
>
> I tried:
>
> No --overwrite:
>
>    # perf record -e syscalls:*enter_nanosleep* usleep 1
>    [ perf record: Woken up 1 times to write data ]
>    [ perf record: Captured and wrote 0.019 MB perf.data (1 samples) ]
>    # perf evlist -v
>    syscalls:sys_enter_nanosleep: type: 2, size: 112, config: 0x132, { sample_period, sample_freq }: 1, sample_type: IP|TID|TIME|CPU|PERIOD|RAW, disabled: 1, inherit: 1, mmap: 1, comm: 1, enable_on_exec: 1, task: 1, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1
>    # Tip: use 'perf evlist --trace-fields' to show fields for tracepoint events
>    # perf script
>            usleep 29416 [002] 220099.782982: syscalls:sys_enter_nanosleep: rqtp: 0x7ffc21f73cc0, rmtp: 0x00000000
>
> Now I went on to try this new --overwrite thing:
>
>    # perf record --overwrite -e syscalls:*enter_nanosleep* usleep 1
>    [ perf record: Woken up 1 times to write data ]
>    [ perf record: Captured and wrote 0.019 MB perf.data ]
>    # perf evlist -v
> syscalls:sys_enter_nanosleep: type: 2, size: 112, config: 0x132, { sample_period, sample_freq }: 1, sample_type: IP|TID|TIME|CPU|PERIOD|RAW, disabled: 1, inherit: 1, mmap: 1, comm: 1, enable_on_exec: 1, task: 1, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1, write_backward: 1
>    # Tip: use 'perf evlist --trace-fields' to show fields for tracepoint events
>    # perf script
>    #
>
> So it hasn't recorded anything at anytime, i.e. I expected, based on the
> documentation provided, that it would get what was in its buffer, to be written,
> i.e. the single "syscalls:sys_enter_nanosleep" event that took place in that
> workload.
>
> So I'm now trying it together with --switch-output, but I just get one
> timestamp suffixed perf.data file, empty, without that event that I know took
> place.
>
> Care to ellaborate here?

Sorry, you need to apply patch 3/7 - 5/7 to enable operations described 
here.

I'll reorder these patches and send again, but some important patch 
would become
untestable until patch 3/7 get applied. I'll try to add a test case for 
them.

Thank you.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web