Groups > linux.kernel > #1427103 > unrolled thread

[GIT PULL 00/10] perf/core improvements and fixes

Started by	Arnaldo Carvalho de Melo <acme@kernel.org>
First post	2016-06-21 00:30 +0200
Last post	2016-06-21 18:20 +0200
Articles	14 — 6 participants

Back to article view | Back to linux.kernel

  [GIT PULL 00/10] perf/core improvements and fixes Arnaldo Carvalho de Melo <acme@kernel.org> - 2016-06-21 00:30 +0200
    [PATCH 04/10] perf hists: Rename __hists__add_entry to hists__add_entry Arnaldo Carvalho de Melo <acme@kernel.org> - 2016-06-21 00:30 +0200
    [PATCH 03/10] perf script: Fix documentation of '-f' when it should be '-F' Arnaldo Carvalho de Melo <acme@kernel.org> - 2016-06-21 00:30 +0200
    [PATCH 01/10] perf tools: Find vdso with the consider of cross-platform Arnaldo Carvalho de Melo <acme@kernel.org> - 2016-06-21 00:30 +0200
    [PATCH 02/10] kbuild: List libelf-devel as an alternative Arnaldo Carvalho de Melo <acme@kernel.org> - 2016-06-21 00:30 +0200
    [PATCH 05/10] perf tools: Remove some unused functions Arnaldo Carvalho de Melo <acme@kernel.org> - 2016-06-21 00:30 +0200
    [PATCH 10/10] perf script: Add stackcollapse.py script Arnaldo Carvalho de Melo <acme@kernel.org> - 2016-06-21 00:30 +0200
      Re: [PATCH 10/10] perf script: Add stackcollapse.py script Brendan Gregg <brendan.d.gregg@gmail.com> - 2016-06-21 20:40 +0200
        Re: [PATCH 10/10] perf script: Add stackcollapse.py script Arnaldo Carvalho de Melo <acme@kernel.org> - 2016-06-21 22:40 +0200
    [PATCH 08/10] perf record: Add --dry-run option to check cmdline options Arnaldo Carvalho de Melo <acme@kernel.org> - 2016-06-21 00:30 +0200
    Re: [GIT PULL 00/10] perf/core improvements and fixes Namhyung Kim <namhyung@kernel.org> - 2016-06-21 04:20 +0200
      Re: [GIT PULL 00/10] perf/core improvements and fixes Brendan Gregg <bgregg@netflix.com> - 2016-06-21 05:20 +0200
        Re: [GIT PULL 00/10] perf/core improvements and fixes Paolo Bonzini <pbonzini@redhat.com> - 2016-06-21 12:20 +0200
          Re: [GIT PULL 00/10] perf/core improvements and fixes Arnaldo Carvalho de Melo <acme@redhat.com> - 2016-06-21 18:20 +0200

#1427103 — [GIT PULL 00/10] perf/core improvements and fixes

From	Arnaldo Carvalho de Melo <acme@kernel.org>
Date	2016-06-21 00:30 +0200
Subject	[GIT PULL 00/10] perf/core improvements and fixes
Message-ID	<rMfRE-5Ht-13@gated-at.bofh.it>

Hi Ingo,

	Please consider pulling,

- Arnaldo

The following changes since commit 02469a95096a549508c5adf61d84a1d72851c85b:

  Merge tag 'perf-core-for-mingo-20160615' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/core (2016-06-16 10:27:35 +0200)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-core-for-mingo-20160620

for you to fetch changes up to 2a0a7c72702bac1b87cd4d49196a334483386fab:

  perf script: Add stackcollapse.py script (2016-06-20 17:50:39 -0300)

----------------------------------------------------------------
perf/core improvements and fixes:

New features:

. Add --dry-run option to 'perf record' to check if command line options can be
  parsed, but not doing any recording (Wang Nan)

. Allow dumping the object files generated by llvm when processing eBPF
  scriptlet events (Wang Nan)

- Add stackcollapse.py script to help generating flame graphs (Paolo Bonzini)

Documentation:

. Fix 'perf script'  documentation of '-f' when it should be '-F' (Adrian Hunter)

Infrastructure:

- Fix write_backwards fallback when using a new tool on older kernels
  without support for this feature (Arnaldo Carvalho de Melo)

- Find vdso file in cross-platform perf.data file processing scenarios (He Kuang)

- Remove some leftovers from the initial codebase copying from git
  (Arnaldo Carvalho de Melo)

- List libelf-devel as an alternative, as this is how the libelf
  development package is called on OpenSuSE (Jean Delvare)

- Rename __hists__add_entry to hists__add_entry (Jiri Olsa)

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

----------------------------------------------------------------
Adrian Hunter (1):
      perf script: Fix documentation of '-f' when it should be '-F'

Arnaldo Carvalho de Melo (3):
      perf tools: Remove some unused functions
      perf tools: Remove --perf-dir and --work-dir
      perf evsel: Fix write_backwards fallback

He Kuang (1):
      perf tools: Find vdso with the consider of cross-platform

Jean Delvare (1):
      kbuild: List libelf-devel as an alternative

Jiri Olsa (1):
      perf hists: Rename __hists__add_entry to hists__add_entry

Paolo Bonzini (1):
      perf script: Add stackcollapse.py script

Wang Nan (2):
      perf llvm: Allow dump llvm output object file using llvm.dump-obj
      perf record: Add --dry-run option to check cmdline options

 Makefile                                           |   2 +-
 tools/perf/Documentation/perf-record.txt           |   7 ++
 tools/perf/Documentation/perf-script.txt           |  20 ++--
 tools/perf/builtin-annotate.c                      |   2 +-
 tools/perf/builtin-diff.c                          |  12 +-
 tools/perf/builtin-record.c                        |   7 ++
 tools/perf/config/Makefile                         |   2 +-
 tools/perf/perf.c                                  |  36 ------
 tools/perf/scripts/python/bin/stackcollapse-record |   8 ++
 tools/perf/scripts/python/bin/stackcollapse-report |   3 +
 tools/perf/scripts/python/stackcollapse.py         | 127 +++++++++++++++++++++
 tools/perf/tests/hists_link.c                      |   4 +-
 tools/perf/util/cache.h                            |   9 --
 tools/perf/util/config.c                           |   2 +-
 tools/perf/util/evsel.c                            |  23 ++--
 tools/perf/util/hist.c                             |  34 +++---
 tools/perf/util/hist.h                             |  14 +--
 tools/perf/util/llvm-utils.c                       |  42 +++++++
 tools/perf/util/llvm-utils.h                       |   5 +
 tools/perf/util/path.c                             |  65 +----------
 tools/perf/util/vdso.c                             |  40 ++++++-
 21 files changed, 288 insertions(+), 176 deletions(-)
 create mode 100755 tools/perf/scripts/python/bin/stackcollapse-record
 create mode 100755 tools/perf/scripts/python/bin/stackcollapse-report
 create mode 100755 tools/perf/scripts/python/stackcollapse.py

[toc] | [next] | [standalone]

#1427105 — [PATCH 04/10] perf hists: Rename histsadd_entry to hists__add_entry

From	Arnaldo Carvalho de Melo <acme@kernel.org>
Date	2016-06-21 00:30 +0200
Subject	[PATCH 04/10] perf hists: Rename __hists__add_entry to hists__add_entry
Message-ID	<rMfRE-5Ht-37@gated-at.bofh.it>
In reply to	#1427103

From: Jiri Olsa <jolsa@kernel.org>

There's no reason we should suffer the '__' prefix for the base global
function.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1465928361-2442-12-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/builtin-annotate.c |  2 +-
 tools/perf/builtin-diff.c     | 12 +-----------
 tools/perf/tests/hists_link.c |  4 ++--
 tools/perf/util/hist.c        | 34 +++++++++++++++++-----------------
 tools/perf/util/hist.h        | 14 +++++++-------
 5 files changed, 28 insertions(+), 38 deletions(-)

diff --git a/tools/perf/builtin-annotate.c b/tools/perf/builtin-annotate.c
index 25c81734a950..a2324e1892aa 100644
--- a/tools/perf/builtin-annotate.c
+++ b/tools/perf/builtin-annotate.c
@@ -75,7 +75,7 @@ static int perf_evsel__add_sample(struct perf_evsel *evsel,
 	sample->period = 1;
 	sample->weight = 1;
 
-	he = __hists__add_entry(hists, al, NULL, NULL, NULL, sample, true);
+	he = hists__add_entry(hists, al, NULL, NULL, NULL, sample, true);
 	if (he == NULL)
 		return -ENOMEM;
 
diff --git a/tools/perf/builtin-diff.c b/tools/perf/builtin-diff.c
index 7f628f9c2fb4..8b6735f35179 100644
--- a/tools/perf/builtin-diff.c
+++ b/tools/perf/builtin-diff.c
@@ -310,16 +310,6 @@ static int formula_fprintf(struct hist_entry *he, struct hist_entry *pair,
 	return -1;
 }
 
-static int hists__add_entry(struct hists *hists,
-			    struct addr_location *al,
-			    struct perf_sample *sample)
-{
-	if (__hists__add_entry(hists, al, NULL, NULL, NULL,
-			       sample, true) != NULL)
-		return 0;
-	return -ENOMEM;
-}
-
 static int diff__process_sample_event(struct perf_tool *tool __maybe_unused,
 				      union perf_event *event,
 				      struct perf_sample *sample,
@@ -336,7 +326,7 @@ static int diff__process_sample_event(struct perf_tool *tool __maybe_unused,
 		return -1;
 	}
 
-	if (hists__add_entry(hists, &al, sample)) {
+	if (!hists__add_entry(hists, &al, NULL, NULL, NULL, sample, true)) {
 		pr_warning("problem incrementing symbol period, skipping event\n");
 		goto out_put;
 	}
diff --git a/tools/perf/tests/hists_link.c b/tools/perf/tests/hists_link.c
index acf5a1301c07..6f96ca4d4fc0 100644
--- a/tools/perf/tests/hists_link.c
+++ b/tools/perf/tests/hists_link.c
@@ -84,7 +84,7 @@ static int add_hist_entries(struct perf_evlist *evlist, struct machine *machine)
 			if (machine__resolve(machine, &al, &sample) < 0)
 				goto out;
 
-			he = __hists__add_entry(hists, &al, NULL,
+			he = hists__add_entry(hists, &al, NULL,
 						NULL, NULL, &sample, true);
 			if (he == NULL) {
 				addr_location__put(&al);
@@ -103,7 +103,7 @@ static int add_hist_entries(struct perf_evlist *evlist, struct machine *machine)
 			if (machine__resolve(machine, &al, &sample) < 0)
 				goto out;
 
-			he = __hists__add_entry(hists, &al, NULL,
+			he = hists__add_entry(hists, &al, NULL,
 						NULL, NULL, &sample, true);
 			if (he == NULL) {
 				addr_location__put(&al);
diff --git a/tools/perf/util/hist.c b/tools/perf/util/hist.c
index 2515cfdb7365..d2647b1d82c0 100644
--- a/tools/perf/util/hist.c
+++ b/tools/perf/util/hist.c
@@ -531,13 +531,13 @@ out:
 	return he;
 }
 
-struct hist_entry *__hists__add_entry(struct hists *hists,
-				      struct addr_location *al,
-				      struct symbol *sym_parent,
-				      struct branch_info *bi,
-				      struct mem_info *mi,
-				      struct perf_sample *sample,
-				      bool sample_self)
+struct hist_entry *hists__add_entry(struct hists *hists,
+				    struct addr_location *al,
+				    struct symbol *sym_parent,
+				    struct branch_info *bi,
+				    struct mem_info *mi,
+				    struct perf_sample *sample,
+				    bool sample_self)
 {
 	struct hist_entry entry = {
 		.thread	= al->thread,
@@ -622,8 +622,8 @@ iter_add_single_mem_entry(struct hist_entry_iter *iter, struct addr_location *al
 	 */
 	sample->period = cost;
 
-	he = __hists__add_entry(hists, al, iter->parent, NULL, mi,
-				sample, true);
+	he = hists__add_entry(hists, al, iter->parent, NULL, mi,
+			      sample, true);
 	if (!he)
 		return -ENOMEM;
 
@@ -727,8 +727,8 @@ iter_add_next_branch_entry(struct hist_entry_iter *iter, struct addr_location *a
 	sample->period = 1;
 	sample->weight = bi->flags.cycles ? bi->flags.cycles : 1;
 
-	he = __hists__add_entry(hists, al, iter->parent, &bi[i], NULL,
-				sample, true);
+	he = hists__add_entry(hists, al, iter->parent, &bi[i], NULL,
+			      sample, true);
 	if (he == NULL)
 		return -ENOMEM;
 
@@ -764,8 +764,8 @@ iter_add_single_normal_entry(struct hist_entry_iter *iter, struct addr_location
 	struct perf_sample *sample = iter->sample;
 	struct hist_entry *he;
 
-	he = __hists__add_entry(evsel__hists(evsel), al, iter->parent, NULL, NULL,
-				sample, true);
+	he = hists__add_entry(evsel__hists(evsel), al, iter->parent, NULL, NULL,
+			      sample, true);
 	if (he == NULL)
 		return -ENOMEM;
 
@@ -825,8 +825,8 @@ iter_add_single_cumulative_entry(struct hist_entry_iter *iter,
 	struct hist_entry *he;
 	int err = 0;
 
-	he = __hists__add_entry(hists, al, iter->parent, NULL, NULL,
-				sample, true);
+	he = hists__add_entry(hists, al, iter->parent, NULL, NULL,
+			      sample, true);
 	if (he == NULL)
 		return -ENOMEM;
 
@@ -900,8 +900,8 @@ iter_add_next_cumulative_entry(struct hist_entry_iter *iter,
 		}
 	}
 
-	he = __hists__add_entry(evsel__hists(evsel), al, iter->parent, NULL, NULL,
-				sample, false);
+	he = hists__add_entry(evsel__hists(evsel), al, iter->parent, NULL, NULL,
+			      sample, false);
 	if (he == NULL)
 		return -ENOMEM;
 
diff --git a/tools/perf/util/hist.h b/tools/perf/util/hist.h
index a19112872ff9..0a03e08be503 100644
--- a/tools/perf/util/hist.h
+++ b/tools/perf/util/hist.h
@@ -120,13 +120,13 @@ extern const struct hist_iter_ops hist_iter_branch;
 extern const struct hist_iter_ops hist_iter_mem;
 extern const struct hist_iter_ops hist_iter_cumulative;
 
-struct hist_entry *__hists__add_entry(struct hists *hists,
-				      struct addr_location *al,
-				      struct symbol *parent,
-				      struct branch_info *bi,
-				      struct mem_info *mi,
-				      struct perf_sample *sample,
-				      bool sample_self);
+struct hist_entry *hists__add_entry(struct hists *hists,
+				    struct addr_location *al,
+				    struct symbol *parent,
+				    struct branch_info *bi,
+				    struct mem_info *mi,
+				    struct perf_sample *sample,
+				    bool sample_self);
 int hist_entry_iter__add(struct hist_entry_iter *iter, struct addr_location *al,
 			 int max_stack_depth, void *arg);
 
-- 
2.5.5

[toc] | [prev] | [next] | [standalone]

#1427106 — [PATCH 03/10] perf script: Fix documentation of '-f' when it should be '-F'

From	Arnaldo Carvalho de Melo <acme@kernel.org>
Date	2016-06-21 00:30 +0200
Subject	[PATCH 03/10] perf script: Fix documentation of '-f' when it should be '-F'
Message-ID	<rMfRE-5Ht-39@gated-at.bofh.it>
In reply to	#1427103

From: Adrian Hunter <adrian.hunter@intel.com>

The documentation for perf script mixes up '-f' and '-F'. Fix it.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/None
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/Documentation/perf-script.txt | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/tools/perf/Documentation/perf-script.txt b/tools/perf/Documentation/perf-script.txt
index 4fc44c75263f..4f34379ebd77 100644
--- a/tools/perf/Documentation/perf-script.txt
+++ b/tools/perf/Documentation/perf-script.txt
@@ -119,13 +119,13 @@ OPTIONS
 	srcline, period, iregs, brstack, brstacksym, flags.
         Field list can be prepended with the type, trace, sw or hw,
         to indicate to which event type the field list applies.
-        e.g., -f sw:comm,tid,time,ip,sym  and -f trace:time,cpu,trace
+        e.g., -F sw:comm,tid,time,ip,sym  and -F trace:time,cpu,trace
 
-		perf script -f <fields>
+		perf script -F <fields>
 
 	is equivalent to:
 
-		perf script -f trace:<fields> -f sw:<fields> -f hw:<fields>
+		perf script -F trace:<fields> -F sw:<fields> -F hw:<fields>
 
 	i.e., the specified fields apply to all event types if the type string
 	is not given.
@@ -133,9 +133,9 @@ OPTIONS
 	The arguments are processed in the order received. A later usage can
 	reset a prior request. e.g.:
 
-		-f trace: -f comm,tid,time,ip,sym
+		-F trace: -F comm,tid,time,ip,sym
 
-	The first -f suppresses trace events (field list is ""), but then the
+	The first -F suppresses trace events (field list is ""), but then the
 	second invocation sets the fields to comm,tid,time,ip,sym. In this case a
 	warning is given to the user:
 
@@ -143,9 +143,9 @@ OPTIONS
 
 	Alternatively, consider the order:
 
-		-f comm,tid,time,ip,sym -f trace:
+		-F comm,tid,time,ip,sym -F trace:
 
-	The first -f sets the fields for all events and the second -f
+	The first -F sets the fields for all events and the second -F
 	suppresses trace events. The user is given a warning message about
 	the override, and the result of the above is that only S/W and H/W
 	events are displayed with the given fields.
@@ -154,14 +154,14 @@ OPTIONS
 	event type, a message is displayed to the user that the option is
 	ignored for that type. For example:
 
-		$ perf script -f comm,tid,trace
+		$ perf script -F comm,tid,trace
 		'trace' not valid for hardware events. Ignoring.
 		'trace' not valid for software events. Ignoring.
 
 	Alternatively, if the type is given an invalid field is specified it
 	is an error. For example:
 
-        perf script -v -f sw:comm,tid,trace
+        perf script -v -F sw:comm,tid,trace
         'trace' not valid for software events.
 
 	At this point usage is displayed, and perf-script exits.
@@ -173,7 +173,7 @@ OPTIONS
 	respectively.
 
 	Finally, a user may not set fields to none for all event types.
-	i.e., -f "" is not allowed.
+	i.e., -F "" is not allowed.
 
 	The brstack output includes branch related information with raw addresses using the
 	/v/v/v/v/ syntax in the following order:
-- 
2.5.5

[toc] | [prev] | [next] | [standalone]

#1427107 — [PATCH 01/10] perf tools: Find vdso with the consider of cross-platform

From	Arnaldo Carvalho de Melo <acme@kernel.org>
Date	2016-06-21 00:30 +0200
Subject	[PATCH 01/10] perf tools: Find vdso with the consider of cross-platform
Message-ID	<rMfRF-5Ht-47@gated-at.bofh.it>
In reply to	#1427103

From: He Kuang <hekuang@huawei.com>

There's a problem in machine__findnew_vdso(), vdso buildid generated by a
32-bit machine stores it with the name 'vdso', but when processing buildid on a
64-bit machine with the same 'perf.data', perf will search for vdso named as
'vdso32' and get failed.

This patch tries to find the existing dsos in machine->dsos by thread dso_type.
64-bit thread tries to find vdso with name 'vdso', because all 64-bit vdso is
named as that. 32-bit thread first tries to find vdso with name 'vdso32' if
this thread was run on 64-bit machine, if failed, then it tries 'vdso' which
indicates that the thread was run on 32-bit machine when recording.

Committer note:

Additional explanation by Adrian Hunter:

We match maps to builds ids using the file name - consider
machine__findnew_[v]dso() called in map__new().  So in the context of a perf
data file, we consider the file name to be unique.

A vdso map does not have a file name - all we know is that it is vdso.  We look
at the thread to tell if it is 32-bit, 64-bit or x32.  Then we need to get the
build id which has been recorded using short name "[vdso]" or "[vdso32]" or
"[vdsox32]".

The problem is that on a 32-bit machine, we use the name "[vdso]".  If you take
a 32-bit perf data file to a 64-bit machine, it gets hard to figure out if
"[vdso]" is 32-bit or 64-bit.

This patch solves that problem.

Signed-off-by: He Kuang <hekuang@huawei.com>
Acked-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Ekaterina Tumanova <tumanova@linux.vnet.ibm.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Kan Liang <kan.liang@intel.com>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1463475894-163531-1-git-send-email-hekuang@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/vdso.c | 40 +++++++++++++++++++++++++++++++++++++---
 1 file changed, 37 insertions(+), 3 deletions(-)

diff --git a/tools/perf/util/vdso.c b/tools/perf/util/vdso.c
index 44d440da15dc..8f81c415723d 100644
--- a/tools/perf/util/vdso.c
+++ b/tools/perf/util/vdso.c
@@ -134,8 +134,6 @@ static struct dso *__machine__addnew_vdso(struct machine *machine, const char *s
 	return dso;
 }
 
-#if BITS_PER_LONG == 64
-
 static enum dso_type machine__thread_dso_type(struct machine *machine,
 					      struct thread *thread)
 {
@@ -156,6 +154,8 @@ static enum dso_type machine__thread_dso_type(struct machine *machine,
 	return dso_type;
 }
 
+#if BITS_PER_LONG == 64
+
 static int vdso__do_copy_compat(FILE *f, int fd)
 {
 	char buf[4096];
@@ -283,8 +283,38 @@ static int __machine__findnew_vdso_compat(struct machine *machine,
 
 #endif
 
+static struct dso *machine__find_vdso(struct machine *machine,
+				      struct thread *thread)
+{
+	struct dso *dso = NULL;
+	enum dso_type dso_type;
+
+	dso_type = machine__thread_dso_type(machine, thread);
+	switch (dso_type) {
+	case DSO__TYPE_32BIT:
+		dso = __dsos__find(&machine->dsos, DSO__NAME_VDSO32, true);
+		if (!dso) {
+			dso = __dsos__find(&machine->dsos, DSO__NAME_VDSO,
+					   true);
+			if (dso_type != dso__type(dso, machine))
+				dso = NULL;
+		}
+		break;
+	case DSO__TYPE_X32BIT:
+		dso = __dsos__find(&machine->dsos, DSO__NAME_VDSOX32, true);
+		break;
+	case DSO__TYPE_64BIT:
+	case DSO__TYPE_UNKNOWN:
+	default:
+		dso = __dsos__find(&machine->dsos, DSO__NAME_VDSO, true);
+		break;
+	}
+
+	return dso;
+}
+
 struct dso *machine__findnew_vdso(struct machine *machine,
-				  struct thread *thread __maybe_unused)
+				  struct thread *thread)
 {
 	struct vdso_info *vdso_info;
 	struct dso *dso = NULL;
@@ -297,6 +327,10 @@ struct dso *machine__findnew_vdso(struct machine *machine,
 	if (!vdso_info)
 		goto out_unlock;
 
+	dso = machine__find_vdso(machine, thread);
+	if (dso)
+		goto out_unlock;
+
 #if BITS_PER_LONG == 64
 	if (__machine__findnew_vdso_compat(machine, thread, vdso_info, &dso))
 		goto out_unlock;
-- 
2.5.5

[toc] | [prev] | [next] | [standalone]

#1427108 — [PATCH 02/10] kbuild: List libelf-devel as an alternative

From	Arnaldo Carvalho de Melo <acme@kernel.org>
Date	2016-06-21 00:30 +0200
Subject	[PATCH 02/10] kbuild: List libelf-devel as an alternative
Message-ID	<rMfRF-5Ht-49@gated-at.bofh.it>
In reply to	#1427103

From: Jean Delvare <jdelvare@suse.de>

On openSUSE, the libelf development files are in package libelf-devel.

Signed-off-by: Jean Delvare <jdelvare@suse.de>
Acked-by: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Michal Marek <mmarek@suse.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: linux-kbuild@vger.kernel.org
Link: http://lkml.kernel.org/n/tip-s8nyk3pyy2927sd7qp7u42oi@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 Makefile                   | 2 +-
 tools/perf/config/Makefile | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index b409076c7c18..4a1fe88d6839 100644
--- a/Makefile
+++ b/Makefile
@@ -1038,7 +1038,7 @@ ifdef CONFIG_STACK_VALIDATION
   ifeq ($(has_libelf),1)
     objtool_target := tools/objtool FORCE
   else
-    $(warning "Cannot use CONFIG_STACK_VALIDATION, please install libelf-dev or elfutils-libelf-devel")
+    $(warning "Cannot use CONFIG_STACK_VALIDATION, please install libelf-dev, libelf-devel or elfutils-libelf-devel")
     SKIP_STACK_VALIDATION := 1
     export SKIP_STACK_VALIDATION
   endif
diff --git a/tools/perf/config/Makefile b/tools/perf/config/Makefile
index 80018feb99c0..534c81176f6c 100644
--- a/tools/perf/config/Makefile
+++ b/tools/perf/config/Makefile
@@ -257,7 +257,7 @@ else
       LIBC_SUPPORT := 1
     endif
     ifeq ($(LIBC_SUPPORT),1)
-      msg := $(warning No libelf found, disables 'probe' tool and BPF support in 'perf record', please install elfutils-libelf-devel/libelf-dev);
+      msg := $(warning No libelf found, disables 'probe' tool and BPF support in 'perf record', please install libelf-dev, libelf-devel or elfutils-libelf-devel);
 
       NO_LIBELF := 1
       NO_DWARF := 1
-- 
2.5.5

[toc] | [prev] | [next] | [standalone]

#1427109 — [PATCH 05/10] perf tools: Remove some unused functions

From	Arnaldo Carvalho de Melo <acme@kernel.org>
Date	2016-06-21 00:30 +0200
Subject	[PATCH 05/10] perf tools: Remove some unused functions
Message-ID	<rMfRF-5Ht-51@gated-at.bofh.it>
In reply to	#1427103

From: Arnaldo Carvalho de Melo <acme@redhat.com>

Probably are there since the beginning, taken from git but never used.

Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-lr65jeefffjeaywoapps9a6i@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/cache.h  |  4 ---
 tools/perf/util/config.c |  2 +-
 tools/perf/util/path.c   | 65 +-----------------------------------------------
 3 files changed, 2 insertions(+), 69 deletions(-)

diff --git a/tools/perf/util/cache.h b/tools/perf/util/cache.h
index 0d814bb74661..f260040abc90 100644
--- a/tools/perf/util/cache.h
+++ b/tools/perf/util/cache.h
@@ -32,7 +32,6 @@ int perf_config_int(const char *, const char *);
 u64 perf_config_u64(const char *, const char *);
 int perf_config_bool(const char *, const char *);
 int config_error_nonbool(const char *);
-const char *perf_config_dirname(const char *, const char *);
 const char *perf_etc_perfconfig(void);
 
 char *alias_lookup(const char *alias);
@@ -45,9 +44,6 @@ static inline int is_absolute_path(const char *path)
 	return path[0] == '/';
 }
 
-char *strip_path_suffix(const char *path, const char *suffix);
-
 char *mkpath(const char *fmt, ...) __attribute__((format (printf, 1, 2)));
-char *perf_path(const char *fmt, ...) __attribute__((format (printf, 1, 2)));
 
 #endif /* __PERF_CACHE_H */
diff --git a/tools/perf/util/config.c b/tools/perf/util/config.c
index 31e09a4e8862..d15c59267644 100644
--- a/tools/perf/util/config.c
+++ b/tools/perf/util/config.c
@@ -372,7 +372,7 @@ int perf_config_bool(const char *name, const char *value)
 	return !!perf_config_bool_or_int(name, value, &discard);
 }
 
-const char *perf_config_dirname(const char *name, const char *value)
+static const char *perf_config_dirname(const char *name, const char *value)
 {
 	if (!name)
 		return NULL;
diff --git a/tools/perf/util/path.c b/tools/perf/util/path.c
index 3bf6bf82ff2d..cff8bf0f87e8 100644
--- a/tools/perf/util/path.c
+++ b/tools/perf/util/path.c
@@ -14,14 +14,8 @@
 
 static char bad_path[] = "/bad-path/";
 /*
- * Two hacks:
+ * One hack:
  */
-
-static const char *get_perf_dir(void)
-{
-	return ".";
-}
-
 static char *get_pathname(void)
 {
 	static char pathname_array[4][PATH_MAX];
@@ -54,60 +48,3 @@ char *mkpath(const char *fmt, ...)
 		return bad_path;
 	return cleanup_path(pathname);
 }
-
-char *perf_path(const char *fmt, ...)
-{
-	const char *perf_dir = get_perf_dir();
-	char *pathname = get_pathname();
-	va_list args;
-	unsigned len;
-
-	len = strlen(perf_dir);
-	if (len > PATH_MAX-100)
-		return bad_path;
-	memcpy(pathname, perf_dir, len);
-	if (len && perf_dir[len-1] != '/')
-		pathname[len++] = '/';
-	va_start(args, fmt);
-	len += vsnprintf(pathname + len, PATH_MAX - len, fmt, args);
-	va_end(args);
-	if (len >= PATH_MAX)
-		return bad_path;
-	return cleanup_path(pathname);
-}
-
-/* strip arbitrary amount of directory separators at end of path */
-static inline int chomp_trailing_dir_sep(const char *path, int len)
-{
-	while (len && is_dir_sep(path[len - 1]))
-		len--;
-	return len;
-}
-
-/*
- * If path ends with suffix (complete path components), returns the
- * part before suffix (sans trailing directory separators).
- * Otherwise returns NULL.
- */
-char *strip_path_suffix(const char *path, const char *suffix)
-{
-	int path_len = strlen(path), suffix_len = strlen(suffix);
-
-	while (suffix_len) {
-		if (!path_len)
-			return NULL;
-
-		if (is_dir_sep(path[path_len - 1])) {
-			if (!is_dir_sep(suffix[suffix_len - 1]))
-				return NULL;
-			path_len = chomp_trailing_dir_sep(path, path_len);
-			suffix_len = chomp_trailing_dir_sep(suffix, suffix_len);
-		}
-		else if (path[--path_len] != suffix[--suffix_len])
-			return NULL;
-	}
-
-	if (path_len && !is_dir_sep(path[path_len - 1]))
-		return NULL;
-	return strndup(path, chomp_trailing_dir_sep(path, path_len));
-}
-- 
2.5.5

[toc] | [prev] | [next] | [standalone]

#1427110 — [PATCH 10/10] perf script: Add stackcollapse.py script

From	Arnaldo Carvalho de Melo <acme@kernel.org>
Date	2016-06-21 00:30 +0200
Subject	[PATCH 10/10] perf script: Add stackcollapse.py script
Message-ID	<rMfRF-5Ht-53@gated-at.bofh.it>
In reply to	#1427103

From: Paolo Bonzini <pbonzini@redhat.com>

Add stackcollapse.py script as an example of parsing call chains, and
also of using optparse to access command line options.

The flame graph tools include a set of scripts that parse output from
various tools (including "perf script"), remove the offsets in the
function and collapse each stack to a single line.  The website also
says "perf report could have a report style [...] that output folded
stacks directly, obviating the need for stackcollapse-perf.pl", so here
it is.

This script is a Python rewrite of stackcollapse-perf.pl, using the perf
scripting interface to access the perf data directly from Python.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Cc: Brendan Gregg <bgregg@netflix.com>
Link: http://lkml.kernel.org/r/1460467573-22989-1-git-send-email-pbonzini@redhat.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/scripts/python/bin/stackcollapse-record |   8 ++
 tools/perf/scripts/python/bin/stackcollapse-report |   3 +
 tools/perf/scripts/python/stackcollapse.py         | 127 +++++++++++++++++++++
 3 files changed, 138 insertions(+)
 create mode 100755 tools/perf/scripts/python/bin/stackcollapse-record
 create mode 100755 tools/perf/scripts/python/bin/stackcollapse-report
 create mode 100755 tools/perf/scripts/python/stackcollapse.py

diff --git a/tools/perf/scripts/python/bin/stackcollapse-record b/tools/perf/scripts/python/bin/stackcollapse-record
new file mode 100755
index 000000000000..9d8f9f0f3a17
--- /dev/null
+++ b/tools/perf/scripts/python/bin/stackcollapse-record
@@ -0,0 +1,8 @@
+#!/bin/sh
+
+#
+# stackcollapse.py can cover all type of perf samples including
+# the tracepoints, so no special record requirements, just record what
+# you want to analyze.
+#
+perf record "$@"
diff --git a/tools/perf/scripts/python/bin/stackcollapse-report b/tools/perf/scripts/python/bin/stackcollapse-report
new file mode 100755
index 000000000000..356b9656393d
--- /dev/null
+++ b/tools/perf/scripts/python/bin/stackcollapse-report
@@ -0,0 +1,3 @@
+#!/bin/sh
+# description: produce callgraphs in short form for scripting use
+perf script -s "$PERF_EXEC_PATH"/scripts/python/stackcollapse.py -- "$@"
diff --git a/tools/perf/scripts/python/stackcollapse.py b/tools/perf/scripts/python/stackcollapse.py
new file mode 100755
index 000000000000..a2dfcda41ae6
--- /dev/null
+++ b/tools/perf/scripts/python/stackcollapse.py
@@ -0,0 +1,127 @@
+#!/usr/bin/perl -w
+#
+# stackcollapse.py - format perf samples with one line per distinct call stack
+#
+# This script's output has two space-separated fields.  The first is a semicolon
+# separated stack including the program name (from the "comm" field) and the
+# function names from the call stack.  The second is a count:
+#
+#  swapper;start_kernel;rest_init;cpu_idle;default_idle;native_safe_halt 2
+#
+# The file is sorted according to the first field.
+#
+# Input may be created and processed using:
+#
+#  perf record -a -g -F 99 sleep 60
+#  perf script report stackcollapse > out.stacks-folded
+#
+# (perf script record stackcollapse works too).
+#
+# Written by Paolo Bonzini <pbonzini@redhat.com>
+# Based on Brendan Gregg's stackcollapse-perf.pl script.
+
+import os
+import sys
+from collections import defaultdict
+from optparse import OptionParser, make_option
+
+sys.path.append(os.environ['PERF_EXEC_PATH'] + \
+                '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
+
+from perf_trace_context import *
+from Core import *
+from EventClass import *
+
+# command line parsing
+
+option_list = [
+    # formatting options for the bottom entry of the stack
+    make_option("--include-tid", dest="include_tid",
+                 action="store_true", default=False,
+                 help="include thread id in stack"),
+    make_option("--include-pid", dest="include_pid",
+                 action="store_true", default=False,
+                 help="include process id in stack"),
+    make_option("--no-comm", dest="include_comm",
+                 action="store_false", default=True,
+                 help="do not separate stacks according to comm"),
+    make_option("--tidy-java", dest="tidy_java",
+                 action="store_true", default=False,
+                 help="beautify Java signatures"),
+    make_option("--kernel", dest="annotate_kernel",
+                 action="store_true", default=False,
+                 help="annotate kernel functions with _[k]")
+]
+
+parser = OptionParser(option_list=option_list)
+(opts, args) = parser.parse_args()
+
+if len(args) != 0:
+    parser.error("unexpected command line argument")
+if opts.include_tid and not opts.include_comm:
+    parser.error("requesting tid but not comm is invalid")
+if opts.include_pid and not opts.include_comm:
+    parser.error("requesting pid but not comm is invalid")
+
+# event handlers
+
+lines = defaultdict(lambda: 0)
+
+def process_event(param_dict):
+    def tidy_function_name(sym, dso):
+        if sym is None:
+            sym = '[unknown]'
+
+        sym = sym.replace(';', ':')
+        if opts.tidy_java:
+            # the original stackcollapse-perf.pl script gives the
+            # example of converting this:
+            #    Lorg/mozilla/javascript/MemberBox;.<init>(Ljava/lang/reflect/Method;)V
+            # to this:
+            #    org/mozilla/javascript/MemberBox:.init
+            sym = sym.replace('<', '')
+            sym = sym.replace('>', '')
+            if sym[0] == 'L' and sym.find('/'):
+                sym = sym[1:]
+            try:
+                sym = sym[:sym.index('(')]
+            except ValueError:
+                pass
+
+        if opts.annotate_kernel and dso == '[kernel.kallsyms]':
+            return sym + '_[k]'
+        else:
+            return sym
+
+    stack = list()
+    if 'callchain' in param_dict:
+        for entry in param_dict['callchain']:
+            entry.setdefault('sym', dict())
+            entry['sym'].setdefault('name', None)
+            entry.setdefault('dso', None)
+            stack.append(tidy_function_name(entry['sym']['name'],
+                                            entry['dso']))
+    else:
+        param_dict.setdefault('symbol', None)
+        param_dict.setdefault('dso', None)
+        stack.append(tidy_function_name(param_dict['symbol'],
+                                        param_dict['dso']))
+
+    if opts.include_comm:
+        comm = param_dict["comm"].replace(' ', '_')
+        sep = "-"
+        if opts.include_pid:
+            comm = comm + sep + str(param_dict['sample']['pid'])
+            sep = "/"
+        if opts.include_tid:
+            comm = comm + sep + str(param_dict['sample']['tid'])
+        stack.append(comm)
+
+    stack_string = ';'.join(reversed(stack))
+    lines[stack_string] = lines[stack_string] + 1
+
+def trace_end():
+    list = lines.keys()
+    list.sort()
+    for stack in list:
+        print "%s %d" % (stack, lines[stack])
-- 
2.5.5

[toc] | [prev] | [next] | [standalone]

#1428061 — Re: [PATCH 10/10] perf script: Add stackcollapse.py script

From	Brendan Gregg <brendan.d.gregg@gmail.com>
Date	2016-06-21 20:40 +0200
Subject	Re: [PATCH 10/10] perf script: Add stackcollapse.py script
Message-ID	<rMyKC-10p-13@gated-at.bofh.it>
In reply to	#1427110

On Mon, Jun 20, 2016 at 3:23 PM, Arnaldo Carvalho de Melo
<acme@kernel.org> wrote:
> From: Paolo Bonzini <pbonzini@redhat.com>
>
> Add stackcollapse.py script as an example of parsing call chains, and
> also of using optparse to access command line options.
>
> The flame graph tools include a set of scripts that parse output from
> various tools (including "perf script"), remove the offsets in the
> function and collapse each stack to a single line.  The website also
> says "perf report could have a report style [...] that output folded
> stacks directly, obviating the need for stackcollapse-perf.pl", so here
> it is.
>
> This script is a Python rewrite of stackcollapse-perf.pl, using the perf
> scripting interface to access the perf data directly from Python.
>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> Acked-by: Jiri Olsa <jolsa@kernel.org>
> Cc: Brendan Gregg <bgregg@netflix.com>
> Link: http://lkml.kernel.org/r/1460467573-22989-1-git-send-email-pbonzini@redhat.com
> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
> ---
>  tools/perf/scripts/python/bin/stackcollapse-record |   8 ++
>  tools/perf/scripts/python/bin/stackcollapse-report |   3 +
>  tools/perf/scripts/python/stackcollapse.py         | 127 +++++++++++++++++++++
>  3 files changed, 138 insertions(+)
>  create mode 100755 tools/perf/scripts/python/bin/stackcollapse-record
>  create mode 100755 tools/perf/scripts/python/bin/stackcollapse-report
>  create mode 100755 tools/perf/scripts/python/stackcollapse.py
>
> diff --git a/tools/perf/scripts/python/bin/stackcollapse-record b/tools/perf/scripts/python/bin/stackcollapse-record
> new file mode 100755
> index 000000000000..9d8f9f0f3a17
> --- /dev/null
> +++ b/tools/perf/scripts/python/bin/stackcollapse-record
> @@ -0,0 +1,8 @@
> +#!/bin/sh
> +
> +#
> +# stackcollapse.py can cover all type of perf samples including
> +# the tracepoints, so no special record requirements, just record what
> +# you want to analyze.
> +#
> +perf record "$@"
> diff --git a/tools/perf/scripts/python/bin/stackcollapse-report b/tools/perf/scripts/python/bin/stackcollapse-report
> new file mode 100755
> index 000000000000..356b9656393d
> --- /dev/null
> +++ b/tools/perf/scripts/python/bin/stackcollapse-report
> @@ -0,0 +1,3 @@
> +#!/bin/sh
> +# description: produce callgraphs in short form for scripting use
> +perf script -s "$PERF_EXEC_PATH"/scripts/python/stackcollapse.py -- "$@"
> diff --git a/tools/perf/scripts/python/stackcollapse.py b/tools/perf/scripts/python/stackcollapse.py
> new file mode 100755
> index 000000000000..a2dfcda41ae6
> --- /dev/null
> +++ b/tools/perf/scripts/python/stackcollapse.py
> @@ -0,0 +1,127 @@
> +#!/usr/bin/perl -w

Perl? I guess this line is ignored when invoked.

Rest looks good. Glad it supports tidy_java code, and kernel annotations.

Brendan

> +#
> +# stackcollapse.py - format perf samples with one line per distinct call stack
> +#
> +# This script's output has two space-separated fields.  The first is a semicolon
> +# separated stack including the program name (from the "comm" field) and the
> +# function names from the call stack.  The second is a count:
> +#
> +#  swapper;start_kernel;rest_init;cpu_idle;default_idle;native_safe_halt 2
> +#
> +# The file is sorted according to the first field.
> +#
> +# Input may be created and processed using:
> +#
> +#  perf record -a -g -F 99 sleep 60
> +#  perf script report stackcollapse > out.stacks-folded
> +#
> +# (perf script record stackcollapse works too).
> +#
> +# Written by Paolo Bonzini <pbonzini@redhat.com>
> +# Based on Brendan Gregg's stackcollapse-perf.pl script.
> +
> +import os
> +import sys
> +from collections import defaultdict
> +from optparse import OptionParser, make_option
> +
> +sys.path.append(os.environ['PERF_EXEC_PATH'] + \
> +                '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
> +
> +from perf_trace_context import *
> +from Core import *
> +from EventClass import *
> +
> +# command line parsing
> +
> +option_list = [
> +    # formatting options for the bottom entry of the stack
> +    make_option("--include-tid", dest="include_tid",
> +                 action="store_true", default=False,
> +                 help="include thread id in stack"),
> +    make_option("--include-pid", dest="include_pid",
> +                 action="store_true", default=False,
> +                 help="include process id in stack"),
> +    make_option("--no-comm", dest="include_comm",
> +                 action="store_false", default=True,
> +                 help="do not separate stacks according to comm"),
> +    make_option("--tidy-java", dest="tidy_java",
> +                 action="store_true", default=False,
> +                 help="beautify Java signatures"),
> +    make_option("--kernel", dest="annotate_kernel",
> +                 action="store_true", default=False,
> +                 help="annotate kernel functions with _[k]")
> +]
> +
> +parser = OptionParser(option_list=option_list)
> +(opts, args) = parser.parse_args()
> +
> +if len(args) != 0:
> +    parser.error("unexpected command line argument")
> +if opts.include_tid and not opts.include_comm:
> +    parser.error("requesting tid but not comm is invalid")
> +if opts.include_pid and not opts.include_comm:
> +    parser.error("requesting pid but not comm is invalid")
> +
> +# event handlers
> +
> +lines = defaultdict(lambda: 0)
> +
> +def process_event(param_dict):
> +    def tidy_function_name(sym, dso):
> +        if sym is None:
> +            sym = '[unknown]'
> +
> +        sym = sym.replace(';', ':')
> +        if opts.tidy_java:
> +            # the original stackcollapse-perf.pl script gives the
> +            # example of converting this:
> +            #    Lorg/mozilla/javascript/MemberBox;.<init>(Ljava/lang/reflect/Method;)V
> +            # to this:
> +            #    org/mozilla/javascript/MemberBox:.init
> +            sym = sym.replace('<', '')
> +            sym = sym.replace('>', '')
> +            if sym[0] == 'L' and sym.find('/'):
> +                sym = sym[1:]
> +            try:
> +                sym = sym[:sym.index('(')]
> +            except ValueError:
> +                pass
> +
> +        if opts.annotate_kernel and dso == '[kernel.kallsyms]':
> +            return sym + '_[k]'
> +        else:
> +            return sym
> +
> +    stack = list()
> +    if 'callchain' in param_dict:
> +        for entry in param_dict['callchain']:
> +            entry.setdefault('sym', dict())
> +            entry['sym'].setdefault('name', None)
> +            entry.setdefault('dso', None)
> +            stack.append(tidy_function_name(entry['sym']['name'],
> +                                            entry['dso']))
> +    else:
> +        param_dict.setdefault('symbol', None)
> +        param_dict.setdefault('dso', None)
> +        stack.append(tidy_function_name(param_dict['symbol'],
> +                                        param_dict['dso']))
> +
> +    if opts.include_comm:
> +        comm = param_dict["comm"].replace(' ', '_')
> +        sep = "-"
> +        if opts.include_pid:
> +            comm = comm + sep + str(param_dict['sample']['pid'])
> +            sep = "/"
> +        if opts.include_tid:
> +            comm = comm + sep + str(param_dict['sample']['tid'])
> +        stack.append(comm)
> +
> +    stack_string = ';'.join(reversed(stack))
> +    lines[stack_string] = lines[stack_string] + 1
> +
> +def trace_end():
> +    list = lines.keys()
> +    list.sort()
> +    for stack in list:
> +        print "%s %d" % (stack, lines[stack])

[toc] | [prev] | [next] | [standalone]

#1428141 — Re: [PATCH 10/10] perf script: Add stackcollapse.py script

From	Arnaldo Carvalho de Melo <acme@kernel.org>
Date	2016-06-21 22:40 +0200
Subject	Re: [PATCH 10/10] perf script: Add stackcollapse.py script
Message-ID	<rMACK-2cL-25@gated-at.bofh.it>
In reply to	#1428061

Em Tue, Jun 21, 2016 at 11:36:10AM -0700, Brendan Gregg escreveu:
> On Mon, Jun 20, 2016 at 3:23 PM, Arnaldo Carvalho de Melo
> <acme@kernel.org> wrote:
> > From: Paolo Bonzini <pbonzini@redhat.com>
> >
> > Add stackcollapse.py script as an example of parsing call chains, and
> > also of using optparse to access command line options.
> >
> > The flame graph tools include a set of scripts that parse output from
> > various tools (including "perf script"), remove the offsets in the
> > function and collapse each stack to a single line.  The website also
> > says "perf report could have a report style [...] that output folded
> > stacks directly, obviating the need for stackcollapse-perf.pl", so here
> > it is.
> >
> > This script is a Python rewrite of stackcollapse-perf.pl, using the perf
> > scripting interface to access the perf data directly from Python.
> >
> > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> > Acked-by: Jiri Olsa <jolsa@kernel.org>
> > Cc: Brendan Gregg <bgregg@netflix.com>
> > Link: http://lkml.kernel.org/r/1460467573-22989-1-git-send-email-pbonzini@redhat.com
> > Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
> > ---
> >  tools/perf/scripts/python/bin/stackcollapse-record |   8 ++
> >  tools/perf/scripts/python/bin/stackcollapse-report |   3 +
> >  tools/perf/scripts/python/stackcollapse.py         | 127 +++++++++++++++++++++
> >  3 files changed, 138 insertions(+)
> >  create mode 100755 tools/perf/scripts/python/bin/stackcollapse-record
> >  create mode 100755 tools/perf/scripts/python/bin/stackcollapse-report
> >  create mode 100755 tools/perf/scripts/python/stackcollapse.py
> >
> > diff --git a/tools/perf/scripts/python/bin/stackcollapse-record b/tools/perf/scripts/python/bin/stackcollapse-record
> > new file mode 100755
> > index 000000000000..9d8f9f0f3a17
> > --- /dev/null
> > +++ b/tools/perf/scripts/python/bin/stackcollapse-record
> > @@ -0,0 +1,8 @@
> > +#!/bin/sh
> > +
> > +#
> > +# stackcollapse.py can cover all type of perf samples including
> > +# the tracepoints, so no special record requirements, just record what
> > +# you want to analyze.
> > +#
> > +perf record "$@"
> > diff --git a/tools/perf/scripts/python/bin/stackcollapse-report b/tools/perf/scripts/python/bin/stackcollapse-report
> > new file mode 100755
> > index 000000000000..356b9656393d
> > --- /dev/null
> > +++ b/tools/perf/scripts/python/bin/stackcollapse-report
> > @@ -0,0 +1,3 @@
> > +#!/bin/sh
> > +# description: produce callgraphs in short form for scripting use
> > +perf script -s "$PERF_EXEC_PATH"/scripts/python/stackcollapse.py -- "$@"
> > diff --git a/tools/perf/scripts/python/stackcollapse.py b/tools/perf/scripts/python/stackcollapse.py
> > new file mode 100755
> > index 000000000000..a2dfcda41ae6
> > --- /dev/null
> > +++ b/tools/perf/scripts/python/stackcollapse.py
> > @@ -0,0 +1,127 @@
> > +#!/usr/bin/perl -w
> 
> Perl? I guess this line is ignored when invoked.

Bummer, will fix and add another check on my .git/hooks/pre-commit
 
> Rest looks good. Glad it supports tidy_java code, and kernel annotations.

Glad you liked it,

- Arnaldo
 
> Brendan
> 
> > +#
> > +# stackcollapse.py - format perf samples with one line per distinct call stack
> > +#
> > +# This script's output has two space-separated fields.  The first is a semicolon
> > +# separated stack including the program name (from the "comm" field) and the
> > +# function names from the call stack.  The second is a count:
> > +#
> > +#  swapper;start_kernel;rest_init;cpu_idle;default_idle;native_safe_halt 2
> > +#
> > +# The file is sorted according to the first field.
> > +#
> > +# Input may be created and processed using:
> > +#
> > +#  perf record -a -g -F 99 sleep 60
> > +#  perf script report stackcollapse > out.stacks-folded
> > +#
> > +# (perf script record stackcollapse works too).
> > +#
> > +# Written by Paolo Bonzini <pbonzini@redhat.com>
> > +# Based on Brendan Gregg's stackcollapse-perf.pl script.
> > +
> > +import os
> > +import sys
> > +from collections import defaultdict
> > +from optparse import OptionParser, make_option
> > +
> > +sys.path.append(os.environ['PERF_EXEC_PATH'] + \
> > +                '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
> > +
> > +from perf_trace_context import *
> > +from Core import *
> > +from EventClass import *
> > +
> > +# command line parsing
> > +
> > +option_list = [
> > +    # formatting options for the bottom entry of the stack
> > +    make_option("--include-tid", dest="include_tid",
> > +                 action="store_true", default=False,
> > +                 help="include thread id in stack"),
> > +    make_option("--include-pid", dest="include_pid",
> > +                 action="store_true", default=False,
> > +                 help="include process id in stack"),
> > +    make_option("--no-comm", dest="include_comm",
> > +                 action="store_false", default=True,
> > +                 help="do not separate stacks according to comm"),
> > +    make_option("--tidy-java", dest="tidy_java",
> > +                 action="store_true", default=False,
> > +                 help="beautify Java signatures"),
> > +    make_option("--kernel", dest="annotate_kernel",
> > +                 action="store_true", default=False,
> > +                 help="annotate kernel functions with _[k]")
> > +]
> > +
> > +parser = OptionParser(option_list=option_list)
> > +(opts, args) = parser.parse_args()
> > +
> > +if len(args) != 0:
> > +    parser.error("unexpected command line argument")
> > +if opts.include_tid and not opts.include_comm:
> > +    parser.error("requesting tid but not comm is invalid")
> > +if opts.include_pid and not opts.include_comm:
> > +    parser.error("requesting pid but not comm is invalid")
> > +
> > +# event handlers
> > +
> > +lines = defaultdict(lambda: 0)
> > +
> > +def process_event(param_dict):
> > +    def tidy_function_name(sym, dso):
> > +        if sym is None:
> > +            sym = '[unknown]'
> > +
> > +        sym = sym.replace(';', ':')
> > +        if opts.tidy_java:
> > +            # the original stackcollapse-perf.pl script gives the
> > +            # example of converting this:
> > +            #    Lorg/mozilla/javascript/MemberBox;.<init>(Ljava/lang/reflect/Method;)V
> > +            # to this:
> > +            #    org/mozilla/javascript/MemberBox:.init
> > +            sym = sym.replace('<', '')
> > +            sym = sym.replace('>', '')
> > +            if sym[0] == 'L' and sym.find('/'):
> > +                sym = sym[1:]
> > +            try:
> > +                sym = sym[:sym.index('(')]
> > +            except ValueError:
> > +                pass
> > +
> > +        if opts.annotate_kernel and dso == '[kernel.kallsyms]':
> > +            return sym + '_[k]'
> > +        else:
> > +            return sym
> > +
> > +    stack = list()
> > +    if 'callchain' in param_dict:
> > +        for entry in param_dict['callchain']:
> > +            entry.setdefault('sym', dict())
> > +            entry['sym'].setdefault('name', None)
> > +            entry.setdefault('dso', None)
> > +            stack.append(tidy_function_name(entry['sym']['name'],
> > +                                            entry['dso']))
> > +    else:
> > +        param_dict.setdefault('symbol', None)
> > +        param_dict.setdefault('dso', None)
> > +        stack.append(tidy_function_name(param_dict['symbol'],
> > +                                        param_dict['dso']))
> > +
> > +    if opts.include_comm:
> > +        comm = param_dict["comm"].replace(' ', '_')
> > +        sep = "-"
> > +        if opts.include_pid:
> > +            comm = comm + sep + str(param_dict['sample']['pid'])
> > +            sep = "/"
> > +        if opts.include_tid:
> > +            comm = comm + sep + str(param_dict['sample']['tid'])
> > +        stack.append(comm)
> > +
> > +    stack_string = ';'.join(reversed(stack))
> > +    lines[stack_string] = lines[stack_string] + 1
> > +
> > +def trace_end():
> > +    list = lines.keys()
> > +    list.sort()
> > +    for stack in list:
> > +        print "%s %d" % (stack, lines[stack])

[toc] | [prev] | [next] | [standalone]

#1427111 — [PATCH 08/10] perf record: Add --dry-run option to check cmdline options

From	Arnaldo Carvalho de Melo <acme@kernel.org>
Date	2016-06-21 00:30 +0200
Subject	[PATCH 08/10] perf record: Add --dry-run option to check cmdline options
Message-ID	<rMfRF-5Ht-57@gated-at.bofh.it>
In reply to	#1427103

From: Wang Nan <wangnan0@huawei.com>

With '--dry-run', 'perf record' doesn't do reall recording. Combine with
llvm.dump-obj option, --dry-run can be used to help compile BPF objects
for embedded platform.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1466064161-48553-3-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/Documentation/perf-record.txt | 7 +++++++
 tools/perf/builtin-record.c              | 7 +++++++
 2 files changed, 14 insertions(+)

diff --git a/tools/perf/Documentation/perf-record.txt b/tools/perf/Documentation/perf-record.txt
index 8dbee832abd9..5b46b1d1a37c 100644
--- a/tools/perf/Documentation/perf-record.txt
+++ b/tools/perf/Documentation/perf-record.txt
@@ -360,6 +360,13 @@ particular perf.data snapshot should be kept or not.
 
 Implies --timestamp-filename, --no-buildid and --no-buildid-cache.
 
+--dry-run::
+Parse options then exit. --dry-run can be used to detect errors in cmdline
+options.
+
+'perf record --dry-run -e' can act as a BPF script compiler if llvm.dump-obj
+in config file is set to true.
+
 SEE ALSO
 --------
 linkperf:perf-stat[1], linkperf:perf-list[1]
diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index d4cf1b0c88f9..b1304ebc8779 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -1274,6 +1274,8 @@ static struct record record = {
 const char record_callchain_help[] = CALLCHAIN_RECORD_HELP
 	"\n\t\t\t\tDefault: fp";
 
+static bool dry_run;
+
 /*
  * XXX Will stay a global variable till we fix builtin-script.c to stop messing
  * with it and switch to use the library functions in perf_evlist that came
@@ -1393,6 +1395,8 @@ struct option __record_options[] = {
 		    "append timestamp to output filename"),
 	OPT_BOOLEAN(0, "switch-output", &record.switch_output,
 		    "Switch output when receive SIGUSR2"),
+	OPT_BOOLEAN(0, "dry-run", &dry_run,
+		    "Parse options then exit"),
 	OPT_END()
 };
 
@@ -1462,6 +1466,9 @@ int cmd_record(int argc, const char **argv, const char *prefix __maybe_unused)
 	if (err)
 		return err;
 
+	if (dry_run)
+		return 0;
+
 	err = bpf__setup_stdout(rec->evlist);
 	if (err) {
 		bpf__strerror_setup_stdout(rec->evlist, err, errbuf, sizeof(errbuf));
-- 
2.5.5

[toc] | [prev] | [next] | [standalone]

#1427214

From	Namhyung Kim <namhyung@kernel.org>
Date	2016-06-21 04:20 +0200
Message-ID	<rMjsd-85d-1@gated-at.bofh.it>
In reply to	#1427103

Hi Arnaldo,

On Mon, Jun 20, 2016 at 07:23:20PM -0300, Arnaldo Carvalho de Melo wrote:
> Hi Ingo,
> 
> 	Please consider pulling,
> 
> - Arnaldo
> 
> The following changes since commit 02469a95096a549508c5adf61d84a1d72851c85b:
> 
>   Merge tag 'perf-core-for-mingo-20160615' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/core (2016-06-16 10:27:35 +0200)
> 
> are available in the git repository at:
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-core-for-mingo-20160620
> 
> for you to fetch changes up to 2a0a7c72702bac1b87cd4d49196a334483386fab:
> 
>   perf script: Add stackcollapse.py script (2016-06-20 17:50:39 -0300)
> 
> ----------------------------------------------------------------
> perf/core improvements and fixes:
> 
> New features:
> 
> . Add --dry-run option to 'perf record' to check if command line options can be
>   parsed, but not doing any recording (Wang Nan)
> 
> . Allow dumping the object files generated by llvm when processing eBPF
>   scriptlet events (Wang Nan)
> 
> - Add stackcollapse.py script to help generating flame graphs (Paolo Bonzini)

I think this is already done by '-g folded'.  Please see:

  http://www.brendangregg.com/blog/2016-04-30/linux-perf-folded.html

Thanks,
Namhyung


> 
> Documentation:
> 
> . Fix 'perf script'  documentation of '-f' when it should be '-F' (Adrian Hunter)
> 
> Infrastructure:
> 
> - Fix write_backwards fallback when using a new tool on older kernels
>   without support for this feature (Arnaldo Carvalho de Melo)
> 
> - Find vdso file in cross-platform perf.data file processing scenarios (He Kuang)
> 
> - Remove some leftovers from the initial codebase copying from git
>   (Arnaldo Carvalho de Melo)
> 
> - List libelf-devel as an alternative, as this is how the libelf
>   development package is called on OpenSuSE (Jean Delvare)
> 
> - Rename __hists__add_entry to hists__add_entry (Jiri Olsa)
> 
> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
> 
> ----------------------------------------------------------------
> Adrian Hunter (1):
>       perf script: Fix documentation of '-f' when it should be '-F'
> 
> Arnaldo Carvalho de Melo (3):
>       perf tools: Remove some unused functions
>       perf tools: Remove --perf-dir and --work-dir
>       perf evsel: Fix write_backwards fallback
> 
> He Kuang (1):
>       perf tools: Find vdso with the consider of cross-platform
> 
> Jean Delvare (1):
>       kbuild: List libelf-devel as an alternative
> 
> Jiri Olsa (1):
>       perf hists: Rename __hists__add_entry to hists__add_entry
> 
> Paolo Bonzini (1):
>       perf script: Add stackcollapse.py script
> 
> Wang Nan (2):
>       perf llvm: Allow dump llvm output object file using llvm.dump-obj
>       perf record: Add --dry-run option to check cmdline options
> 
>  Makefile                                           |   2 +-
>  tools/perf/Documentation/perf-record.txt           |   7 ++
>  tools/perf/Documentation/perf-script.txt           |  20 ++--
>  tools/perf/builtin-annotate.c                      |   2 +-
>  tools/perf/builtin-diff.c                          |  12 +-
>  tools/perf/builtin-record.c                        |   7 ++
>  tools/perf/config/Makefile                         |   2 +-
>  tools/perf/perf.c                                  |  36 ------
>  tools/perf/scripts/python/bin/stackcollapse-record |   8 ++
>  tools/perf/scripts/python/bin/stackcollapse-report |   3 +
>  tools/perf/scripts/python/stackcollapse.py         | 127 +++++++++++++++++++++
>  tools/perf/tests/hists_link.c                      |   4 +-
>  tools/perf/util/cache.h                            |   9 --
>  tools/perf/util/config.c                           |   2 +-
>  tools/perf/util/evsel.c                            |  23 ++--
>  tools/perf/util/hist.c                             |  34 +++---
>  tools/perf/util/hist.h                             |  14 +--
>  tools/perf/util/llvm-utils.c                       |  42 +++++++
>  tools/perf/util/llvm-utils.h                       |   5 +
>  tools/perf/util/path.c                             |  65 +----------
>  tools/perf/util/vdso.c                             |  40 ++++++-
>  21 files changed, 288 insertions(+), 176 deletions(-)
>  create mode 100755 tools/perf/scripts/python/bin/stackcollapse-record
>  create mode 100755 tools/perf/scripts/python/bin/stackcollapse-report
>  create mode 100755 tools/perf/scripts/python/stackcollapse.py

[toc] | [prev] | [next] | [standalone]

#1427227

From	Brendan Gregg <bgregg@netflix.com>
Date	2016-06-21 05:20 +0200
Message-ID	<rMkoi-cA-7@gated-at.bofh.it>
In reply to	#1427214

On Mon, Jun 20, 2016 at 7:18 PM, Namhyung Kim <namhyung@kernel.org> wrote:
>
> Hi Arnaldo,
>
> On Mon, Jun 20, 2016 at 07:23:20PM -0300, Arnaldo Carvalho de Melo wrote:
> > Hi Ingo,
> >
> >       Please consider pulling,
> >
> > - Arnaldo
> >
> > The following changes since commit 02469a95096a549508c5adf61d84a1d72851c85b:
> >
> >   Merge tag 'perf-core-for-mingo-20160615' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/core (2016-06-16 10:27:35 +0200)
> >
> > are available in the git repository at:
> >
> >   git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-core-for-mingo-20160620
> >
> > for you to fetch changes up to 2a0a7c72702bac1b87cd4d49196a334483386fab:
> >
> >   perf script: Add stackcollapse.py script (2016-06-20 17:50:39 -0300)
> >
> > ----------------------------------------------------------------
> > perf/core improvements and fixes:
> >
> > New features:
> >
> > . Add --dry-run option to 'perf record' to check if command line options can be
> >   parsed, but not doing any recording (Wang Nan)
> >
> > . Allow dumping the object files generated by llvm when processing eBPF
> >   scriptlet events (Wang Nan)
> >
> > - Add stackcollapse.py script to help generating flame graphs (Paolo Bonzini)
>
> I think this is already done by '-g folded'.  Please see:
>
>   http://www.brendangregg.com/blog/2016-04-30/linux-perf-folded.html
>

Pretty much. Two similar solutions were developed around the same
time. Although I have to use some awk to get "perf -g folded" in the
exact right format, and stackcollapse-perf.py does that directly.

Brendan

Brendan Gregg, Senior Performance Architect, Netflix

[toc] | [prev] | [next] | [standalone]

#1427565

From	Paolo Bonzini <pbonzini@redhat.com>
Date	2016-06-21 12:20 +0200
Message-ID	<rMqWJ-4rb-15@gated-at.bofh.it>
In reply to	#1427227


On 21/06/2016 05:11, Brendan Gregg wrote:
>>> > > - Add stackcollapse.py script to help generating flame graphs (Paolo Bonzini)
>> >
>> > I think this is already done by '-g folded'.  Please see:
>> >
>> >   http://www.brendangregg.com/blog/2016-04-30/linux-perf-folded.html
>> >
> Pretty much. Two similar solutions were developed around the same
> time. Although I have to use some awk to get "perf -g folded" in the
> exact right format, and stackcollapse-perf.py does that directly.

Yes, the idea for stackcollapse-perf.py was:

- to do everything directly and emit "the right" format for the
flamegraph tools.  This however is a very minimal output, and it's not
necessarily the best for perf in general.

- to provide an example of visiting callgraphs from the scripting
interface, since it's not really documented.  From the commit message:
"Add stackcollapse.py script as an example of parsing call chains, and
also of using optparse to access command line options".

Paolo

[toc] | [prev] | [next] | [standalone]

#1427916

From	Arnaldo Carvalho de Melo <acme@redhat.com>
Date	2016-06-21 18:20 +0200
Message-ID	<rMwz8-88j-17@gated-at.bofh.it>
In reply to	#1427565

Em Tue, Jun 21, 2016 at 12:08:47PM +0200, Paolo Bonzini escreveu:
> On 21/06/2016 05:11, Brendan Gregg wrote:
> >>> > > - Add stackcollapse.py script to help generating flame graphs (Paolo Bonzini)
> >> >
> >> > I think this is already done by '-g folded'.  Please see:
> >> >
> >> >   http://www.brendangregg.com/blog/2016-04-30/linux-perf-folded.html
> >> >
> > Pretty much. Two similar solutions were developed around the same
> > time. Although I have to use some awk to get "perf -g folded" in the
> > exact right format, and stackcollapse-perf.py does that directly.
> 
> Yes, the idea for stackcollapse-perf.py was:
> 
> - to do everything directly and emit "the right" format for the
> flamegraph tools.  This however is a very minimal output, and it's not
> necessarily the best for perf in general.
> 
> - to provide an example of visiting callgraphs from the scripting
> interface, since it's not really documented.  From the commit message:
> "Add stackcollapse.py script as an example of parsing call chains, and
> also of using optparse to access command line options".

Yeah, agreed, its short, and it provides the output ready to be used, no
need for extra processing, up to the user to decide if one or the other
is more suitable to their needs.

As a bonus, it documents callgraphs in scripts, good deal :-)

- Arnaldo

[toc] | [prev] | [standalone]

csiph-web

[GIT PULL 00/10] perf/core improvements and fixes

Contents

#1427103 — [GIT PULL 00/10] perf/core improvements and fixes

#1427105 — [PATCH 04/10] perf hists: Rename __hists__add_entry to hists__add_entry

#1427106 — [PATCH 03/10] perf script: Fix documentation of '-f' when it should be '-F'

#1427107 — [PATCH 01/10] perf tools: Find vdso with the consider of cross-platform

#1427108 — [PATCH 02/10] kbuild: List libelf-devel as an alternative

#1427109 — [PATCH 05/10] perf tools: Remove some unused functions

#1427110 — [PATCH 10/10] perf script: Add stackcollapse.py script

#1428061 — Re: [PATCH 10/10] perf script: Add stackcollapse.py script

#1428141 — Re: [PATCH 10/10] perf script: Add stackcollapse.py script

#1427111 — [PATCH 08/10] perf record: Add --dry-run option to check cmdline options

#1427214

#1427227

#1427565

#1427916

#1427105 — [PATCH 04/10] perf hists: Rename histsadd_entry to hists__add_entry