Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1288272 > unrolled thread

[PATCHSET 00/16] perf top: Add multi-thread support (v1)

Back to article view | Back to linux.kernel

Contents

  [PATCHSET 00/16] perf top: Add multi-thread support (v1) Namhyung Kim <namhyung@kernel.org> - 2015-12-10 09:00 +0100
    [PATCH/RFC 03/16] perf top: Factor out warnings about kernel addresses and symbols Namhyung Kim <namhyung@kernel.org> - 2015-12-10 09:00 +0100
      Re: [PATCH/RFC 03/16] perf top: Factor out warnings about kernel  addresses and symbols Arnaldo Carvalho de Melo <acme@kernel.org> - 2015-12-10 20:10 +0100
    [PATCH/RFC 14/16] perf top: Separate struct perf_top_stats Namhyung Kim <namhyung@kernel.org> - 2015-12-10 09:00 +0100
    [PATCH/RFC 11/16] perf top: Implement basic parallel processing Namhyung Kim <namhyung@kernel.org> - 2015-12-10 09:00 +0100
    [PATCH/RFC 01/16] perf top: Delete half-processed hist entries when exit Namhyung Kim <namhyung@kernel.org> - 2015-12-10 09:00 +0100
      RE: [PATCH/RFC 01/16] perf top: Delete half-processed hist entries  when exit 平松雅巳 / HIRAMATU，MASAMI   <masami.hiramatsu.pt@hitachi.com> - 2015-12-10 11:00 +0100
        Re: [PATCH/RFC 01/16] perf top: Delete half-processed hist entries  when exit Arnaldo Carvalho de Melo <acme@kernel.org> - 2015-12-10 20:00 +0100
    [PATCH/RFC 10/16] perf hist: Add events_stats__add() and hists__add_stats() Namhyung Kim <namhyung@kernel.org> - 2015-12-10 09:00 +0100
    [PATCH/RFC 16/16] perf tools: Skip dso front cache for multi-threaded lookup Namhyung Kim <namhyung@kernel.org> - 2015-12-10 09:00 +0100
    [PATCH/RFC 04/16] perf top: Factor out warnings in perf_top__record_precise_ip() Namhyung Kim <namhyung@kernel.org> - 2015-12-10 09:00 +0100
    [PATCH/RFC 15/16] perf top: Add --num-thread option Namhyung Kim <namhyung@kernel.org> - 2015-12-10 09:00 +0100
    [PATCH/RFC 09/16] perf tools: Update hist entry's hists pointer Namhyung Kim <namhyung@kernel.org> - 2015-12-10 09:00 +0100
    Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Ingo Molnar <mingo@kernel.org> - 2015-12-10 09:10 +0100
      Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Namhyung Kim <namhyung@gmail.com> - 2015-12-10 09:50 +0100
        Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Ingo Molnar <mingo@kernel.org> - 2015-12-11 09:20 +0100
          Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) David Ahern <dsahern@gmail.com> - 2015-12-11 16:10 +0100

#1288272 — [PATCHSET 00/16] perf top: Add multi-thread support (v1)

Hello,

This patchset if an attempt to support multi-threading in perf top.
In fact, perf top already run on two threads - a worker thread and a
display thread.  However processing all samples with a single thread
in a large machine can have scalability problems.

This patchset extends it to have multiple worker threads to process
samples concurrently.  Users can control the number of threads using
 --num-thread option.  And there's a collector thread for passing hist
entries from worker threads to the display thread.

This basically has same concept of my previous work with perf report
multi-thread support [1].  I decided to work on perf top first, since
it requires smaller changes.  If this work finishes with a good
result, I'll apply it to perf report as well, and continue to work on
it.  So please test (especially on large machines) and give feedbacks. :)

First 6 patches are fixes and cleanups which can be applied separately.
Rest implements multi-thread support and improves it.

You can get it from 'perf/top-threaded-v1' branch in my tree

  git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git

Any comments are welcome!
Thanks,
Namhyung


[1] https://lkml.org/lkml/2015/10/2/16


Namhyung Kim (16):
  perf top: Delete half-processed hist entries when exit
  perf top: Fix and cleanup perf_top__record_precise_ip()
  perf top: Factor out warnings about kernel addresses and symbols
  perf top: Factor out warnings in perf_top__record_precise_ip()
  perf top: Show warning messages in the display thread
  perf top: Get rid of access to hists->lock in perf_top__record_precise_ip()
  perf hists: Pass hists struct to hist_entry_iter struct
  perf tools: Export a couple of hist functions
  perf tools: Update hist entry's hists pointer
  perf hist: Add events_stats__add() and hists__add_stats()
  perf top: Implement basic parallel processing
  perf tools: Reduce lock contention when processing events
  perf top: Protect the seen list using mutex
  perf top: Separate struct perf_top_stats
  perf top: Add --num-thread option
  perf tools: Skip dso front cache for multi-threaded lookup

 tools/perf/builtin-report.c       |   1 +
 tools/perf/builtin-top.c          | 491 +++++++++++++++++++++++++++++---------
 tools/perf/tests/hists_cumulate.c |   1 +
 tools/perf/tests/hists_filter.c   |   1 +
 tools/perf/tests/hists_output.c   |   1 +
 tools/perf/util/event.c           |   7 +-
 tools/perf/util/hist.c            |  98 ++++++--
 tools/perf/util/hist.h            |  12 +
 tools/perf/util/machine.c         |  19 +-
 tools/perf/util/symbol.c          |   3 +-
 tools/perf/util/symbol.h          |   3 +-
 tools/perf/util/top.c             |  18 +-
 tools/perf/util/top.h             |  14 +-
 13 files changed, 514 insertions(+), 155 deletions(-)

-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1288273 — [PATCH/RFC 03/16] perf top: Factor out warnings about kernel addresses and symbols

Factor out warning messages into separate functions.  These will be
called in the display thread later.

Signed-off-by: Namhyung Kim <namhyung@kernel.org>
---
 tools/perf/builtin-top.c | 95 ++++++++++++++++++++++++++----------------------
 1 file changed, 51 insertions(+), 44 deletions(-)

diff --git a/tools/perf/builtin-top.c b/tools/perf/builtin-top.c
index 7cd9bb69f5a6..e6166ef8fd1a 100644
--- a/tools/perf/builtin-top.c
+++ b/tools/perf/builtin-top.c
@@ -170,6 +170,53 @@ static void ui__warn_map_erange(struct map *map, struct symbol *sym, u64 ip)
 	map->erange_warned = true;
 }
 
+static void ui__warn_kptr_restrict(struct perf_top *top, struct addr_location *al)
+{
+	if (!top->kptr_restrict_warned) {
+		ui__warning(
+"Kernel address maps (/proc/{kallsyms,modules}) are restricted.\n\n"
+"Check /proc/sys/kernel/kptr_restrict.\n\n"
+"Kernel%s samples will not be resolved.\n",
+			  al->map && !RB_EMPTY_ROOT(&al->map->dso->symbols[MAP__FUNCTION]) ?
+			  " modules" : "");
+		if (use_browser <= 0)
+			sleep(5);
+		top->kptr_restrict_warned = true;
+	}
+}
+
+static void ui__warn_vmlinux(struct perf_top *top, struct addr_location *al)
+{
+	const char *msg = "Kernel samples will not be resolved.\n";
+	/*
+	 * As we do lazy loading of symtabs we only will know if the
+	 * specified vmlinux file is invalid when we actually have a
+	 * hit in kernel space and then try to load it. So if we get
+	 * here and there are _no_ symbols in the DSO backing the
+	 * kernel map, bail out.
+	 *
+	 * We may never get here, for instance, if we use -K/
+	 * --hide-kernel-symbols, even if the user specifies an
+	 * invalid --vmlinux ;-)
+	 */
+	if (!top->kptr_restrict_warned && !top->vmlinux_warned &&
+	    RB_EMPTY_ROOT(&al->map->dso->symbols[MAP__FUNCTION])) {
+		if (symbol_conf.vmlinux_name) {
+			char serr[256];
+			dso__strerror_load(al->map->dso, serr, sizeof(serr));
+			ui__warning("The %s file can't be used: %s\n%s",
+				    symbol_conf.vmlinux_name, serr, msg);
+		} else {
+			ui__warning("A vmlinux file was not found.\n%s",
+				    msg);
+		}
+
+		if (use_browser <= 0)
+			sleep(5);
+		top->vmlinux_warned = true;
+	}
+}
+
 static void perf_top__record_precise_ip(struct perf_top *top,
 					struct hist_entry *he,
 					int counter, u64 ip)
@@ -729,51 +776,11 @@ static void perf_event__process_sample(struct perf_tool *tool,
 	if (perf_event__preprocess_sample(event, machine, &al, sample) < 0)
 		return;
 
-	if (!top->kptr_restrict_warned &&
-	    symbol_conf.kptr_restrict &&
-	    al.cpumode == PERF_RECORD_MISC_KERNEL) {
-		ui__warning(
-"Kernel address maps (/proc/{kallsyms,modules}) are restricted.\n\n"
-"Check /proc/sys/kernel/kptr_restrict.\n\n"
-"Kernel%s samples will not be resolved.\n",
-			  al.map && !RB_EMPTY_ROOT(&al.map->dso->symbols[MAP__FUNCTION]) ?
-			  " modules" : "");
-		if (use_browser <= 0)
-			sleep(5);
-		top->kptr_restrict_warned = true;
-	}
-
-	if (al.sym == NULL) {
-		const char *msg = "Kernel samples will not be resolved.\n";
-		/*
-		 * As we do lazy loading of symtabs we only will know if the
-		 * specified vmlinux file is invalid when we actually have a
-		 * hit in kernel space and then try to load it. So if we get
-		 * here and there are _no_ symbols in the DSO backing the
-		 * kernel map, bail out.
-		 *
-		 * We may never get here, for instance, if we use -K/
-		 * --hide-kernel-symbols, even if the user specifies an
-		 * invalid --vmlinux ;-)
-		 */
-		if (!top->kptr_restrict_warned && !top->vmlinux_warned &&
-		    al.map == machine->vmlinux_maps[MAP__FUNCTION] &&
-		    RB_EMPTY_ROOT(&al.map->dso->symbols[MAP__FUNCTION])) {
-			if (symbol_conf.vmlinux_name) {
-				char serr[256];
-				dso__strerror_load(al.map->dso, serr, sizeof(serr));
-				ui__warning("The %s file can't be used: %s\n%s",
-					    symbol_conf.vmlinux_name, serr, msg);
-			} else {
-				ui__warning("A vmlinux file was not found.\n%s",
-					    msg);
-			}
+	if (symbol_conf.kptr_restrict && al.cpumode == PERF_RECORD_MISC_KERNEL)
+		ui__warn_kptr_restrict(top, &al);
 
-			if (use_browser <= 0)
-				sleep(5);
-			top->vmlinux_warned = true;
-		}
-	}
+	if (al.sym == NULL && al.map == machine->vmlinux_maps[MAP__FUNCTION])
+		ui__warn_vmlinux(top, &al);
 
 	if (al.sym == NULL || !al.sym->ignore) {
 		struct hists *hists = evsel__hists(evsel);
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1288851 — Re: [PATCH/RFC 03/16] perf top: Factor out warnings about kernel addresses and symbols

Em Thu, Dec 10, 2015 at 04:53:22PM +0900, Namhyung Kim escreveu:
> Factor out warning messages into separate functions.  These will be
> called in the display thread later.
> 
> Signed-off-by: Namhyung Kim <namhyung@kernel.org>
> ---
>  tools/perf/builtin-top.c | 95 ++++++++++++++++++++++++++----------------------
>  1 file changed, 51 insertions(+), 44 deletions(-)

Without applying patch 2, will check if it happens without this patch as well



[root@ssdandy ~]# echo 2 > /proc/sys/kernel/kptr_restrict
[root@ssdandy ~]# perf top
perf: Segmentation fault
-------- backtrace --------
perf[0x538b3b]
/lib64/libc.so.6(+0x35650)[0x7f401c036650]
perf[0x43af66]
perf(cmd_top+0xedf)[0x43cdbf]
perf[0x47b7c3]
perf(main+0x617)[0x4222b7]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f401c022af5]
perf[0x4223c9]
[0x0]
[root@ssdandy ~]# 
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1288274 — [PATCH/RFC 14/16] perf top: Separate struct perf_top_stats

The perf top maintains various stats regarding samples.  Separate out
those stats so that it can be updated concurrently.

Signed-off-by: Namhyung Kim <namhyung@kernel.org>
---
 tools/perf/builtin-top.c | 36 ++++++++++++++++++++++++++++--------
 tools/perf/util/top.c    | 18 ++++++++----------
 tools/perf/util/top.h    | 12 ++++++++----
 3 files changed, 44 insertions(+), 22 deletions(-)

diff --git a/tools/perf/builtin-top.c b/tools/perf/builtin-top.c
index 5987986b5203..f3ab46b234b6 100644
--- a/tools/perf/builtin-top.c
+++ b/tools/perf/builtin-top.c
@@ -902,8 +902,26 @@ struct reader_arg {
 	int			idx;
 	struct perf_top		*top;
 	struct hists		*hists;
+	struct perf_top_stats	stats;
 };
 
+static void perf_top_stats__add(struct perf_top_stats *dst,
+				struct perf_top_stats *src)
+{
+	static pthread_mutex_t stats_lock = PTHREAD_MUTEX_INITIALIZER;
+
+	pthread_mutex_lock(&stats_lock);
+
+	dst->samples              += src->samples;
+	dst->exact_samples        += src->exact_samples;
+	dst->kernel_samples       += src->kernel_samples;
+	dst->us_samples           += src->us_samples;
+	dst->guest_kernel_samples += src->guest_kernel_samples;
+	dst->guest_us_samples     += src->guest_us_samples;
+
+	pthread_mutex_unlock(&stats_lock);
+}
+
 static void perf_event__process_sample(struct reader_arg *rarg,
 				       const union perf_event *event,
 				       struct perf_evsel *evsel,
@@ -938,7 +956,7 @@ static void perf_event__process_sample(struct reader_arg *rarg,
 	}
 
 	if (event->header.misc & PERF_RECORD_MISC_EXACT_IP)
-		top->exact_samples++;
+		rarg->stats.exact_samples++;
 
 	if (perf_event__preprocess_sample(event, machine, &al, sample) < 0)
 		return;
@@ -1000,28 +1018,28 @@ static void perf_top__mmap_read(struct reader_arg *rarg)
 		origin = event->header.misc & PERF_RECORD_MISC_CPUMODE_MASK;
 
 		if (event->header.type == PERF_RECORD_SAMPLE)
-			++top->samples;
+			++rarg->stats.samples;
 
 		switch (origin) {
 		case PERF_RECORD_MISC_USER:
-			++top->us_samples;
+			++rarg->stats.us_samples;
 			if (top->hide_user_symbols)
 				goto next_event;
 			machine = &session->machines.host;
 			break;
 		case PERF_RECORD_MISC_KERNEL:
-			++top->kernel_samples;
+			++rarg->stats.kernel_samples;
 			if (top->hide_kernel_symbols)
 				goto next_event;
 			machine = &session->machines.host;
 			break;
 		case PERF_RECORD_MISC_GUEST_KERNEL:
-			++top->guest_kernel_samples;
+			++rarg->stats.guest_kernel_samples;
 			machine = perf_session__find_machine(session,
 							     sample.pid);
 			break;
 		case PERF_RECORD_MISC_GUEST_USER:
-			++top->guest_us_samples;
+			++rarg->stats.guest_us_samples;
 			/*
 			 * TODO: we don't process guest user from host side
 			 * except simple counting.
@@ -1065,12 +1083,14 @@ static void *mmap_read_worker(void *arg)
 	}
 
 	while (!done) {
-		u64 hits = top->samples;
+		u64 hits = rarg->stats.samples;
 
 		perf_top__mmap_read(rarg);
 
-		if (hits == top->samples)
+		if (hits == rarg->stats.samples)
 			perf_evlist__poll(top->evlist, 100);
+		else
+			perf_top_stats__add(&top->stats, &rarg->stats);
 	}
 	return NULL;
 }
diff --git a/tools/perf/util/top.c b/tools/perf/util/top.c
index 8e517def925b..95d6bba1a2a0 100644
--- a/tools/perf/util/top.c
+++ b/tools/perf/util/top.c
@@ -30,10 +30,10 @@ size_t perf_top__header_snprintf(struct perf_top *top, char *bf, size_t size)
 	struct target *target = &opts->target;
 	size_t ret = 0;
 
-	if (top->samples) {
-		samples_per_sec = top->samples / top->delay_secs;
-		ksamples_per_sec = top->kernel_samples / top->delay_secs;
-		esamples_percent = (100.0 * top->exact_samples) / top->samples;
+	if (top->stats.samples) {
+		samples_per_sec = top->stats.samples / top->delay_secs;
+		ksamples_per_sec = top->stats.kernel_samples / top->delay_secs;
+		esamples_percent = (100.0 * top->stats.exact_samples) / top->stats.samples;
 	} else {
 		samples_per_sec = ksamples_per_sec = esamples_percent = 0.0;
 	}
@@ -49,9 +49,9 @@ size_t perf_top__header_snprintf(struct perf_top *top, char *bf, size_t size)
 			       "  exact: %4.1f%% [", samples_per_sec,
 			       ksamples_percent, esamples_percent);
 	} else {
-		float us_samples_per_sec = top->us_samples / top->delay_secs;
-		float guest_kernel_samples_per_sec = top->guest_kernel_samples / top->delay_secs;
-		float guest_us_samples_per_sec = top->guest_us_samples / top->delay_secs;
+		float us_samples_per_sec = top->stats.us_samples / top->delay_secs;
+		float guest_kernel_samples_per_sec = top->stats.guest_kernel_samples / top->delay_secs;
+		float guest_us_samples_per_sec = top->stats.guest_us_samples / top->delay_secs;
 
 		ret = SNPRINTF(bf, size,
 			       "   PerfTop:%8.0f irqs/sec  kernel:%4.1f%% us:%4.1f%%"
@@ -111,7 +111,5 @@ size_t perf_top__header_snprintf(struct perf_top *top, char *bf, size_t size)
 
 void perf_top__reset_sample_counters(struct perf_top *top)
 {
-	top->samples = top->us_samples = top->kernel_samples =
-	top->exact_samples = top->guest_kernel_samples =
-	top->guest_us_samples = 0;
+	memset(&top->stats, 0, sizeof(top->stats));
 }
diff --git a/tools/perf/util/top.h b/tools/perf/util/top.h
index c56a00cff5b4..55eb5aebae59 100644
--- a/tools/perf/util/top.h
+++ b/tools/perf/util/top.h
@@ -11,18 +11,22 @@ struct perf_evlist;
 struct perf_evsel;
 struct perf_session;
 
+struct perf_top_stats {
+	u64		   samples;
+	u64		   exact_samples;
+	u64		   kernel_samples, us_samples;
+	u64		   guest_kernel_samples, guest_us_samples;
+};
+
 struct perf_top {
 	struct perf_tool   tool;
 	struct perf_evlist *evlist;
 	struct record_opts record_opts;
+	struct perf_top_stats stats;
 	/*
 	 * Symbols will be added here in perf_event__process_sample and will
 	 * get out after decayed.
 	 */
-	u64		   samples;
-	u64		   kernel_samples, us_samples;
-	u64		   exact_samples;
-	u64		   guest_us_samples, guest_kernel_samples;
 	int		   print_entries, count_filter, delay_secs;
 	int		   max_stack;
 	bool		   hide_kernel_symbols, hide_user_symbols, zero;
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1288275 — [PATCH/RFC 11/16] perf top: Implement basic parallel processing

This patch changes perf top to process event samples with multiple
threads.  For now, each mmap is read and processed with its own hists by
dedicated reader threads in parallel.  And then a single collector
thread gathers the hist entries and move it to the evsel's hists tree.
As usual, a single UI thread will display them.

Signed-off-by: Namhyung Kim <namhyung@kernel.org>
---
 tools/perf/builtin-top.c | 172 ++++++++++++++++++++++++++++++++++++++---------
 1 file changed, 141 insertions(+), 31 deletions(-)

diff --git a/tools/perf/builtin-top.c b/tools/perf/builtin-top.c
index b62665ce5ea6..a9b7461be4f0 100644
--- a/tools/perf/builtin-top.c
+++ b/tools/perf/builtin-top.c
@@ -831,6 +831,57 @@ static int symbol_filter(struct map *map, struct symbol *sym)
 	return 0;
 }
 
+struct collector_arg {
+	struct perf_top		*top;
+	struct hists		*hists;
+};
+
+static void collect_hists(struct perf_top *top, struct hists *hists)
+{
+	int i, k;
+	struct perf_evsel *evsel;
+
+	for (i = 0, k = 0; i < top->evlist->nr_mmaps; i++) {
+		evlist__for_each(top->evlist, evsel) {
+			struct hists *src_hists = &hists[k++];
+			struct hists *dst_hists = evsel__hists(evsel);
+			struct hist_entry *he;
+			struct rb_root *root;
+			struct rb_node *next;
+
+			root = hists__get_rotate_entries_in(src_hists);
+			next = rb_first(root);
+
+			while (next) {
+				if (session_done())
+					return;
+				he = rb_entry(next, struct hist_entry, rb_node_in);
+				next = rb_next(next);
+
+				rb_erase(&he->rb_node_in, root);
+
+				pthread_mutex_lock(&dst_hists->lock);
+				hists__collapse_insert_entry(dst_hists,
+							     dst_hists->entries_in, he);
+				pthread_mutex_unlock(&dst_hists->lock);
+			}
+			hists__add_stats(dst_hists, src_hists);
+		}
+	}
+}
+
+static void *collect_worker(void *arg)
+{
+	struct collector_arg *carg = arg;
+
+	while (!done) {
+		collect_hists(carg->top, carg->hists);
+		poll(NULL, 0, 100);
+	}
+
+	return NULL;
+}
+
 static int hist_iter__top_callback(struct hist_entry_iter *iter,
 				   struct addr_location *al, bool single,
 				   void *arg)
@@ -847,13 +898,19 @@ static int hist_iter__top_callback(struct hist_entry_iter *iter,
 	return 0;
 }
 
-static void perf_event__process_sample(struct perf_tool *tool,
+struct reader_arg {
+	int			idx;
+	struct perf_top		*top;
+	struct hists		*hists;
+};
+
+static void perf_event__process_sample(struct reader_arg *rarg,
 				       const union perf_event *event,
 				       struct perf_evsel *evsel,
 				       struct perf_sample *sample,
 				       struct machine *machine)
 {
-	struct perf_top *top = container_of(tool, struct perf_top, tool);
+	struct perf_top *top = rarg->top;
 	struct addr_location al;
 	int err;
 
@@ -890,10 +947,10 @@ static void perf_event__process_sample(struct perf_tool *tool,
 		perf_top__request_warning(top, &al, WARN_VMLINUX);
 
 	if (al.sym == NULL || !al.sym->ignore) {
-		struct hists *hists = evsel__hists(evsel);
+		struct hists* hists = &rarg->hists[evsel->idx];
 		struct hist_entry_iter iter = {
 			.evsel		= evsel,
-			.hists 		= evsel__hists(evsel),
+			.hists 		= hists,
 			.sample 	= sample,
 			.add_entry_cb 	= hist_iter__top_callback,
 		};
@@ -915,13 +972,15 @@ static void perf_event__process_sample(struct perf_tool *tool,
 	addr_location__put(&al);
 }
 
-static void perf_top__mmap_read_idx(struct perf_top *top, int idx)
+static void perf_top__mmap_read(struct reader_arg *rarg)
 {
 	struct perf_sample sample;
 	struct perf_evsel *evsel;
+	struct perf_top *top = rarg->top;
 	struct perf_session *session = top->session;
 	union perf_event *event;
 	struct machine *machine;
+	int idx = rarg->idx;
 	u8 origin;
 	int ret;
 
@@ -974,10 +1033,11 @@ static void perf_top__mmap_read_idx(struct perf_top *top, int idx)
 
 
 		if (event->header.type == PERF_RECORD_SAMPLE) {
-			perf_event__process_sample(&top->tool, event, evsel,
+			perf_event__process_sample(rarg, event, evsel,
 						   &sample, machine);
 		} else if (event->header.type < PERF_RECORD_MAX) {
-			hists__inc_nr_events(evsel__hists(evsel), event->header.type);
+			hists__inc_nr_events(&rarg->hists[evsel->idx],
+					     event->header.type);
 			machine__process_event(machine, event, &sample);
 		} else
 			++session->evlist->stats.nr_unknown_events;
@@ -986,12 +1046,30 @@ static void perf_top__mmap_read_idx(struct perf_top *top, int idx)
 	}
 }
 
-static void perf_top__mmap_read(struct perf_top *top)
+static void *mmap_read_worker(void *arg)
 {
-	int i;
+	struct reader_arg *rarg = arg;
+	struct perf_top *top = rarg->top;
+
+	if (top->realtime_prio) {
+		struct sched_param param;
+
+		param.sched_priority = top->realtime_prio;
+		if (sched_setscheduler(0, SCHED_FIFO, &param)) {
+			ui__error("Could not set realtime priority.\n");
+			return NULL;
+		}
+	}
+
+	while (!done) {
+		u64 hits = top->samples;
 
-	for (i = 0; i < top->evlist->nr_mmaps; i++)
-		perf_top__mmap_read_idx(top, i);
+		perf_top__mmap_read(rarg);
+
+		if (hits == top->samples)
+			perf_evlist__poll(top->evlist, 100);
+	}
+	return NULL;
 }
 
 static int perf_top__start_counters(struct perf_top *top)
@@ -1052,8 +1130,14 @@ static int perf_top__setup_sample_type(struct perf_top *top __maybe_unused)
 static int __cmd_top(struct perf_top *top)
 {
 	struct record_opts *opts = &top->record_opts;
-	pthread_t thread;
+	pthread_t *readers = NULL;
+	pthread_t collector = (pthread_t) 0;
+	pthread_t ui_thread = (pthread_t) 0;
+	struct hists *hists = NULL;
+	struct reader_arg *rargs = NULL;
+	struct collector_arg carg;
 	int ret;
+	int i;
 
 	top->session = perf_session__new(NULL, false, NULL);
 	if (top->session == NULL)
@@ -1104,37 +1188,63 @@ static int __cmd_top(struct perf_top *top)
 	/* Wait for a minimal set of events before starting the snapshot */
 	perf_evlist__poll(top->evlist, 100);
 
-	perf_top__mmap_read(top);
-
 	ret = -1;
-	if (pthread_create(&thread, NULL, (use_browser > 0 ? display_thread_tui :
-							    display_thread), top)) {
-		ui__error("Could not create display thread.\n");
+	readers = calloc(sizeof(pthread_t), top->evlist->nr_mmaps);
+	if (readers == NULL)
 		goto out_delete;
-	}
 
-	if (top->realtime_prio) {
-		struct sched_param param;
+	rargs = calloc(sizeof(*rargs), top->evlist->nr_mmaps);
+	if (rargs == NULL)
+		goto out_free;
 
-		param.sched_priority = top->realtime_prio;
-		if (sched_setscheduler(0, SCHED_FIFO, &param)) {
-			ui__error("Could not set realtime priority.\n");
-			goto out_join;
-		}
+	hists = calloc(sizeof(*hists), top->evlist->nr_mmaps * top->evlist->nr_entries);
+	if (hists == NULL)
+		goto out_free;
+
+	for (i = 0; i < top->evlist->nr_mmaps * top->evlist->nr_entries; i++)
+		__hists__init(&hists[i]);
+
+	for (i = 0; i < top->evlist->nr_mmaps; i++) {
+		struct reader_arg *rarg = &rargs[i];
+
+		rarg->idx = i;
+		rarg->top = top;
+		rarg->hists = &hists[i * top->evlist->nr_entries];
+
+		perf_top__mmap_read(rarg);
 	}
+	collect_hists(top, hists);
 
-	while (!done) {
-		u64 hits = top->samples;
+	for (i = 0; i < top->evlist->nr_mmaps; i++) {
+		if (pthread_create(&readers[i], NULL, mmap_read_worker, &rargs[i]))
+			goto out_join;
+	}
 
-		perf_top__mmap_read(top);
+	carg.top = top;
+	carg.hists = hists;
+	if (pthread_create(&collector, NULL, collect_worker, &carg))
+		goto out_join;
 
-		if (hits == top->samples)
-			ret = perf_evlist__poll(top->evlist, 100);
+	if (pthread_create(&ui_thread, NULL, (use_browser > 0 ? display_thread_tui :
+							        display_thread), top)) {
+		ui__error("Could not create display thread.\n");
+		goto out_join;
 	}
 
 	ret = 0;
+
 out_join:
-	pthread_join(thread, NULL);
+	pthread_join(ui_thread, NULL);
+	pthread_join(collector, NULL);
+	for (i = 0; i < top->evlist->nr_mmaps; i++) {
+		pthread_join(readers[i], NULL);
+	}
+
+out_free:
+	free(hists);
+	free(rargs);
+	free(readers);
+
 out_delete:
 	perf_session__delete(top->session);
 	top->session = NULL;
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1288276 — [PATCH/RFC 01/16] perf top: Delete half-processed hist entries when exit

After sample processing is done, hist entries are in both of
hists->entries and hists->entries_in (or hists->entries_collapsed).
So I guess perf report does not have leaks on hists.

But for perf top, it's possible to have half-processed entries which
are only in hists->entries_in.  Eventually they will go to the
hists->entries and get freed but they cannot be deleted by current
hists__delete_entries().  This patch adds hists__delete_all_entries
function to delete those entries.

Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
---
 tools/perf/util/hist.c | 26 +++++++++++++++++++++++++-
 1 file changed, 25 insertions(+), 1 deletion(-)

diff --git a/tools/perf/util/hist.c b/tools/perf/util/hist.c
index 565ea3549894..56e97f5af598 100644
--- a/tools/perf/util/hist.c
+++ b/tools/perf/util/hist.c
@@ -270,6 +270,8 @@ static void hists__delete_entry(struct hists *hists, struct hist_entry *he)
 
 	if (sort__need_collapse)
 		rb_erase(&he->rb_node_in, &hists->entries_collapsed);
+	else
+		rb_erase(&he->rb_node_in, hists->entries_in);
 
 	--hists->nr_entries;
 	if (!he->filtered)
@@ -1567,11 +1569,33 @@ static int hists_evsel__init(struct perf_evsel *evsel)
 	return 0;
 }
 
+static void hists__delete_remaining_entries(struct rb_root *root)
+{
+	struct rb_node *node;
+	struct hist_entry *he;
+
+	while (!RB_EMPTY_ROOT(root)) {
+		node = rb_first(root);
+		rb_erase(node, root);
+
+		he = rb_entry(node, struct hist_entry, rb_node_in);
+		hist_entry__delete(he);
+	}
+}
+
+static void hists__delete_all_entries(struct hists *hists)
+{
+	hists__delete_entries(hists);
+	hists__delete_remaining_entries(&hists->entries_in_array[0]);
+	hists__delete_remaining_entries(&hists->entries_in_array[1]);
+	hists__delete_remaining_entries(&hists->entries_collapsed);
+}
+
 static void hists_evsel__exit(struct perf_evsel *evsel)
 {
 	struct hists *hists = evsel__hists(evsel);
 
-	hists__delete_entries(hists);
+	hists__delete_all_entries(hists);
 }
 
 /*
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1288413 — RE: [PATCH/RFC 01/16] perf top: Delete half-processed hist entries when exit

PkZyb206IE5hbWh5dW5nIEtpbSBbbWFpbHRvOm5hbWh5dW5nQGtlcm5lbC5vcmddDQo+DQo+QWZ0
ZXIgc2FtcGxlIHByb2Nlc3NpbmcgaXMgZG9uZSwgaGlzdCBlbnRyaWVzIGFyZSBpbiBib3RoIG9m
DQo+aGlzdHMtPmVudHJpZXMgYW5kIGhpc3RzLT5lbnRyaWVzX2luIChvciBoaXN0cy0+ZW50cmll
c19jb2xsYXBzZWQpLg0KPlNvIEkgZ3Vlc3MgcGVyZiByZXBvcnQgZG9lcyBub3QgaGF2ZSBsZWFr
cyBvbiBoaXN0cy4NCj4NCj5CdXQgZm9yIHBlcmYgdG9wLCBpdCdzIHBvc3NpYmxlIHRvIGhhdmUg
aGFsZi1wcm9jZXNzZWQgZW50cmllcyB3aGljaA0KPmFyZSBvbmx5IGluIGhpc3RzLT5lbnRyaWVz
X2luLiAgRXZlbnR1YWxseSB0aGV5IHdpbGwgZ28gdG8gdGhlDQo+aGlzdHMtPmVudHJpZXMgYW5k
IGdldCBmcmVlZCBidXQgdGhleSBjYW5ub3QgYmUgZGVsZXRlZCBieSBjdXJyZW50DQo+aGlzdHNf
X2RlbGV0ZV9lbnRyaWVzKCkuICBUaGlzIHBhdGNoIGFkZHMgaGlzdHNfX2RlbGV0ZV9hbGxfZW50
cmllcw0KPmZ1bmN0aW9uIHRvIGRlbGV0ZSB0aG9zZSBlbnRyaWVzLg0KPg0KDQpUaGlzIGlzIHRl
c3RlZCB1bmRlciB0aGUgcmVmY250IGRlYnVnZ2VyIGFuZCBJJ3ZlIHJldmlld2VkIGl0Lg0KDQpB
Y2tlZC1ieTogTWFzYW1pIEhpcmFtYXRzdSA8bWFzYW1pLmhpcmFtYXRzdS5wdEBoaXRhY2hpLmNv
bT4NClRlc3RlZC1ieTogTWFzYW1pIEhpcmFtYXRzdSA8bWFzYW1pLmhpcmFtYXRzdS5wdEBoaXRh
Y2hpLmNvbT4NCg0KDQpUaGFua3MhDQoNCj5DYzogTWFzYW1pIEhpcmFtYXRzdSA8bWFzYW1pLmhp
cmFtYXRzdS5wdEBoaXRhY2hpLmNvbT4NCj5TaWduZWQtb2ZmLWJ5OiBOYW1oeXVuZyBLaW0gPG5h
bWh5dW5nQGtlcm5lbC5vcmc+DQo+LS0tDQo+IHRvb2xzL3BlcmYvdXRpbC9oaXN0LmMgfCAyNiAr
KysrKysrKysrKysrKysrKysrKysrKysrLQ0KPiAxIGZpbGUgY2hhbmdlZCwgMjUgaW5zZXJ0aW9u
cygrKSwgMSBkZWxldGlvbigtKQ0KPg0KPmRpZmYgLS1naXQgYS90b29scy9wZXJmL3V0aWwvaGlz
dC5jIGIvdG9vbHMvcGVyZi91dGlsL2hpc3QuYw0KPmluZGV4IDU2NWVhMzU0OTg5NC4uNTZlOTdm
NWFmNTk4IDEwMDY0NA0KPi0tLSBhL3Rvb2xzL3BlcmYvdXRpbC9oaXN0LmMNCj4rKysgYi90b29s
cy9wZXJmL3V0aWwvaGlzdC5jDQo+QEAgLTI3MCw2ICsyNzAsOCBAQCBzdGF0aWMgdm9pZCBoaXN0
c19fZGVsZXRlX2VudHJ5KHN0cnVjdCBoaXN0cyAqaGlzdHMsIHN0cnVjdCBoaXN0X2VudHJ5ICpo
ZSkNCj4NCj4gCWlmIChzb3J0X19uZWVkX2NvbGxhcHNlKQ0KPiAJCXJiX2VyYXNlKCZoZS0+cmJf
bm9kZV9pbiwgJmhpc3RzLT5lbnRyaWVzX2NvbGxhcHNlZCk7DQo+KwllbHNlDQo+KwkJcmJfZXJh
c2UoJmhlLT5yYl9ub2RlX2luLCBoaXN0cy0+ZW50cmllc19pbik7DQo+DQo+IAktLWhpc3RzLT5u
cl9lbnRyaWVzOw0KPiAJaWYgKCFoZS0+ZmlsdGVyZWQpDQo+QEAgLTE1NjcsMTEgKzE1NjksMzMg
QEAgc3RhdGljIGludCBoaXN0c19ldnNlbF9faW5pdChzdHJ1Y3QgcGVyZl9ldnNlbCAqZXZzZWwp
DQo+IAlyZXR1cm4gMDsNCj4gfQ0KPg0KPitzdGF0aWMgdm9pZCBoaXN0c19fZGVsZXRlX3JlbWFp
bmluZ19lbnRyaWVzKHN0cnVjdCByYl9yb290ICpyb290KQ0KPit7DQo+KwlzdHJ1Y3QgcmJfbm9k
ZSAqbm9kZTsNCj4rCXN0cnVjdCBoaXN0X2VudHJ5ICpoZTsNCj4rDQo+Kwl3aGlsZSAoIVJCX0VN
UFRZX1JPT1Qocm9vdCkpIHsNCj4rCQlub2RlID0gcmJfZmlyc3Qocm9vdCk7DQo+KwkJcmJfZXJh
c2Uobm9kZSwgcm9vdCk7DQo+Kw0KPisJCWhlID0gcmJfZW50cnkobm9kZSwgc3RydWN0IGhpc3Rf
ZW50cnksIHJiX25vZGVfaW4pOw0KPisJCWhpc3RfZW50cnlfX2RlbGV0ZShoZSk7DQo+Kwl9DQo+
K30NCj4rDQo+K3N0YXRpYyB2b2lkIGhpc3RzX19kZWxldGVfYWxsX2VudHJpZXMoc3RydWN0IGhp
c3RzICpoaXN0cykNCj4rew0KPisJaGlzdHNfX2RlbGV0ZV9lbnRyaWVzKGhpc3RzKTsNCj4rCWhp
c3RzX19kZWxldGVfcmVtYWluaW5nX2VudHJpZXMoJmhpc3RzLT5lbnRyaWVzX2luX2FycmF5WzBd
KTsNCj4rCWhpc3RzX19kZWxldGVfcmVtYWluaW5nX2VudHJpZXMoJmhpc3RzLT5lbnRyaWVzX2lu
X2FycmF5WzFdKTsNCj4rCWhpc3RzX19kZWxldGVfcmVtYWluaW5nX2VudHJpZXMoJmhpc3RzLT5l
bnRyaWVzX2NvbGxhcHNlZCk7DQo+K30NCj4rDQo+IHN0YXRpYyB2b2lkIGhpc3RzX2V2c2VsX19l
eGl0KHN0cnVjdCBwZXJmX2V2c2VsICpldnNlbCkNCj4gew0KPiAJc3RydWN0IGhpc3RzICpoaXN0
cyA9IGV2c2VsX19oaXN0cyhldnNlbCk7DQo+DQo+LQloaXN0c19fZGVsZXRlX2VudHJpZXMoaGlz
dHMpOw0KPisJaGlzdHNfX2RlbGV0ZV9hbGxfZW50cmllcyhoaXN0cyk7DQo+IH0NCj4NCj4gLyoN
Cj4tLQ0KPjIuNi4yDQoNCg==
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1288838 — Re: [PATCH/RFC 01/16] perf top: Delete half-processed hist entries when exit

Em Thu, Dec 10, 2015 at 09:55:44AM +0000, 平松雅巳 / HIRAMATU，MASAMI escreveu:
> >From: Namhyung Kim [mailto:namhyung@kernel.org]
> >
> >After sample processing is done, hist entries are in both of
> >hists->entries and hists->entries_in (or hists->entries_collapsed).
> >So I guess perf report does not have leaks on hists.
> >
> >But for perf top, it's possible to have half-processed entries which
> >are only in hists->entries_in.  Eventually they will go to the
> >hists->entries and get freed but they cannot be deleted by current
> >hists__delete_entries().  This patch adds hists__delete_all_entries
> >function to delete those entries.
> >
> 
> This is tested under the refcnt debugger and I've reviewed it.
> 
> Acked-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
> Tested-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>

Thanks, applied.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1288278 — [PATCH/RFC 10/16] perf hist: Add events_stats__add() and hists__add_stats()

These two functions are to update event and hists stats.  They'll be
used by multi threads to update local stats in the later patch.

Signed-off-by: Namhyung Kim <namhyung@kernel.org>
---
 tools/perf/util/hist.c | 23 +++++++++++++++++++++++
 tools/perf/util/hist.h |  6 ++++++
 2 files changed, 29 insertions(+)

diff --git a/tools/perf/util/hist.c b/tools/perf/util/hist.c
index a12e5022fe04..08396a7fea23 100644
--- a/tools/perf/util/hist.c
+++ b/tools/perf/util/hist.c
@@ -1347,6 +1347,29 @@ void events_stats__inc(struct events_stats *stats, u32 type)
 	++stats->nr_events[type];
 }
 
+void events_stats__add(struct events_stats *dst, struct events_stats *src)
+{
+	int i;
+
+#define ADD(_field)  dst->_field += src->_field
+
+	ADD(total_period);
+	ADD(total_non_filtered_period);
+	ADD(total_lost);
+	ADD(total_invalid_chains);
+	ADD(nr_non_filtered_samples);
+	ADD(nr_lost_warned);
+	ADD(nr_unknown_events);
+	ADD(nr_invalid_chains);
+	ADD(nr_unknown_id);
+	ADD(nr_unprocessable_samples);
+
+	for (i = 0; i < PERF_RECORD_HEADER_MAX; i++)
+		ADD(nr_events[i]);
+
+#undef ADD
+}
+
 void hists__inc_nr_events(struct hists *hists, u32 type)
 {
 	events_stats__inc(&hists->stats, type);
diff --git a/tools/perf/util/hist.h b/tools/perf/util/hist.h
index 670720ef8acd..725afce73738 100644
--- a/tools/perf/util/hist.h
+++ b/tools/perf/util/hist.h
@@ -141,8 +141,14 @@ void hists__inc_stats(struct hists *hists, struct hist_entry *h);
 void hists__inc_nr_events(struct hists *hists, u32 type);
 void hists__inc_nr_samples(struct hists *hists, bool filtered);
 void events_stats__inc(struct events_stats *stats, u32 type);
+void events_stats__add(struct events_stats *dst, struct events_stats *src);
 size_t events_stats__fprintf(struct events_stats *stats, FILE *fp);
 
+static inline void hists__add_stats(struct hists *dst, struct hists *src)
+{
+	events_stats__add(&dst->stats, &src->stats);
+}
+
 size_t hists__fprintf(struct hists *hists, bool show_header, int max_rows,
 		      int max_cols, float min_pcnt, FILE *fp);
 size_t perf_evlist__fprintf_nr_events(struct perf_evlist *evlist, FILE *fp);
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1288279 — [PATCH/RFC 16/16] perf tools: Skip dso front cache for multi-threaded lookup

Currently the dso maintains a front cache for faster symbol lookup,
but access to it should be synchronized when multi thread is used.
Also it doesn't help much if data file has callchains since it should
walk through the callchains for each sample so single cache is almost
meaningless.

So skip the cache if mult-thread is enabled.

Signed-off-by: Namhyung Kim <namhyung@kernel.org>
---
 tools/perf/builtin-top.c | 2 ++
 tools/perf/util/symbol.c | 3 +++
 tools/perf/util/symbol.h | 3 ++-
 3 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/tools/perf/builtin-top.c b/tools/perf/builtin-top.c
index fc9715b046b3..d69069d57f8c 100644
--- a/tools/perf/builtin-top.c
+++ b/tools/perf/builtin-top.c
@@ -1225,6 +1225,8 @@ static int __cmd_top(struct perf_top *top)
 	if ((int)top->nr_threads > top->evlist->nr_mmaps)
 		top->nr_threads = top->evlist->nr_mmaps;
 
+	symbol_conf.multi_thread = (top->nr_threads > 1);
+
 	nr_idx = top->evlist->nr_mmaps / top->nr_threads;
 	rem = top->evlist->nr_mmaps % top->nr_threads;
 
diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
index d51abd2e7865..d4a966004fa0 100644
--- a/tools/perf/util/symbol.c
+++ b/tools/perf/util/symbol.c
@@ -454,6 +454,9 @@ void dso__reset_find_symbol_cache(struct dso *dso)
 struct symbol *dso__find_symbol(struct dso *dso,
 				enum map_type type, u64 addr)
 {
+	if (symbol_conf.multi_thread)
+		return symbols__find(&dso->symbols[type], addr);
+
 	if (dso->last_find_result[type].addr != addr) {
 		dso->last_find_result[type].addr   = addr;
 		dso->last_find_result[type].symbol = symbols__find(&dso->symbols[type], addr);
diff --git a/tools/perf/util/symbol.h b/tools/perf/util/symbol.h
index 857f707ac12b..68c198f64e1d 100644
--- a/tools/perf/util/symbol.h
+++ b/tools/perf/util/symbol.h
@@ -109,7 +109,8 @@ struct symbol_conf {
 			branch_callstack,
 			has_filter,
 			show_ref_callgraph,
-			hide_unresolved;
+			hide_unresolved,
+			multi_thread;
 	const char	*vmlinux_name,
 			*kallsyms_name,
 			*source_prefix,
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1288280 — [PATCH/RFC 04/16] perf top: Factor out warnings in perf_top__record_precise_ip()

Currently it warns two error cases during annotation update.  One is
for ERANGE and it already is in a separate function.  Fix this
function to be consistent with others like checking erange_warned
inside the function and passing 'al' in the argument.

Another case is for ENOMEM, make it also as a separate function.

Signed-off-by: Namhyung Kim <namhyung@kernel.org>
---
 tools/perf/builtin-top.c | 87 +++++++++++++++++++++++++++++-------------------
 tools/perf/util/top.h    |  1 +
 2 files changed, 53 insertions(+), 35 deletions(-)

diff --git a/tools/perf/builtin-top.c b/tools/perf/builtin-top.c
index e6166ef8fd1a..7a237719037a 100644
--- a/tools/perf/builtin-top.c
+++ b/tools/perf/builtin-top.c
@@ -143,31 +143,50 @@ static void __zero_source_counters(struct hist_entry *he)
 	symbol__annotate_zero_histograms(sym);
 }
 
-static void ui__warn_map_erange(struct map *map, struct symbol *sym, u64 ip)
+static void ui__warn_map_erange(struct perf_top *top __maybe_unused,
+				struct addr_location *al)
 {
-	struct utsname uts;
-	int err = uname(&uts);
-
-	ui__warning("Out of bounds address found:\n\n"
-		    "Addr:   %" PRIx64 "\n"
-		    "DSO:    %s %c\n"
-		    "Map:    %" PRIx64 "-%" PRIx64 "\n"
-		    "Symbol: %" PRIx64 "-%" PRIx64 " %c %s\n"
-		    "Arch:   %s\n"
-		    "Kernel: %s\n"
-		    "Tools:  %s\n\n"
-		    "Not all samples will be on the annotation output.\n\n"
-		    "Please report to linux-kernel@vger.kernel.org\n",
-		    ip, map->dso->long_name, dso__symtab_origin(map->dso),
-		    map->start, map->end, sym->start, sym->end,
-		    sym->binding == STB_GLOBAL ? 'g' :
-		    sym->binding == STB_LOCAL  ? 'l' : 'w', sym->name,
-		    err ? "[unknown]" : uts.machine,
-		    err ? "[unknown]" : uts.release, perf_version_string);
-	if (use_browser <= 0)
-		sleep(5);
-
-	map->erange_warned = true;
+	struct map *map = al->map;
+	struct symbol *sym = al->sym;
+	u64 ip = al->addr;
+
+	if (!map->erange_warned) {
+		struct utsname uts;
+		int err = uname(&uts);
+
+		ui__warning("Out of bounds address found:\n\n"
+			    "Addr:   %" PRIx64 "\n"
+			    "DSO:    %s %c\n"
+			    "Map:    %" PRIx64 "-%" PRIx64 "\n"
+			    "Symbol: %" PRIx64 "-%" PRIx64 " %c %s\n"
+			    "Arch:   %s\n"
+			    "Kernel: %s\n"
+			    "Tools:  %s\n\n"
+			    "Not all samples will be on the annotation output.\n\n"
+			    "Please report to linux-kernel@vger.kernel.org\n",
+			    ip, map->dso->long_name, dso__symtab_origin(map->dso),
+			    map->start, map->end, sym->start, sym->end,
+			    sym->binding == STB_GLOBAL ? 'g' :
+			    sym->binding == STB_LOCAL  ? 'l' : 'w', sym->name,
+			    err ? "[unknown]" : uts.machine,
+			    err ? "[unknown]" : uts.release, perf_version_string);
+		if (use_browser <= 0)
+			sleep(5);
+
+		map->erange_warned = true;
+	}
+}
+
+static void ui__warn_enomem(struct perf_top *top, struct addr_location *al)
+{
+	if (!top->enomem_warned) {
+		ui__warning("Not enough memory for annotating '%s' symbol!\n",
+			    al->sym->name);
+
+		if (use_browser <= 0)
+			sleep(5);
+		top->enomem_warned = true;
+	}
 }
 
 static void ui__warn_kptr_restrict(struct perf_top *top, struct addr_location *al)
@@ -219,10 +238,11 @@ static void ui__warn_vmlinux(struct perf_top *top, struct addr_location *al)
 
 static void perf_top__record_precise_ip(struct perf_top *top,
 					struct hist_entry *he,
-					int counter, u64 ip)
+					struct addr_location *al,
+					int counter)
 {
 	struct annotation *notes;
-	struct symbol *sym = he->ms.sym;
+	struct symbol *sym = al->sym;
 	int err = 0;
 
 	if (sym == NULL || (use_browser == 0 &&
@@ -235,7 +255,7 @@ static void perf_top__record_precise_ip(struct perf_top *top,
 	if (pthread_mutex_trylock(&notes->lock))
 		return;
 
-	err = hist_entry__inc_addr_samples(he, counter, ip);
+	err = hist_entry__inc_addr_samples(he, counter, al->addr);
 
 	pthread_mutex_unlock(&notes->lock);
 
@@ -246,13 +266,10 @@ static void perf_top__record_precise_ip(struct perf_top *top,
 		 */
 		pthread_mutex_unlock(&he->hists->lock);
 
-		if (err == -ERANGE && !he->ms.map->erange_warned)
-			ui__warn_map_erange(he->ms.map, sym, ip);
-		else if (err == -ENOMEM) {
-			pr_err("Not enough memory for annotating '%s' symbol!\n",
-			       sym->name);
-			sleep(1);
-		}
+		if (err == -ERANGE)
+			ui__warn_map_erange(top, al);
+		else if (err == -ENOMEM)
+			ui__warn_enomem(top, al);
 
 		pthread_mutex_lock(&he->hists->lock);
 	}
@@ -733,7 +750,7 @@ static int hist_iter__top_callback(struct hist_entry_iter *iter,
 	struct perf_evsel *evsel = iter->evsel;
 
 	if (sort__has_sym && single)
-		perf_top__record_precise_ip(top, he, evsel->idx, al->addr);
+		perf_top__record_precise_ip(top, he, al, evsel->idx);
 
 	hist__account_cycles(iter->sample->branch_stack, al, iter->sample,
 		     !(top->record_opts.branch_stack & PERF_SAMPLE_BRANCH_ANY));
diff --git a/tools/perf/util/top.h b/tools/perf/util/top.h
index f92c37abb0a8..c56a00cff5b4 100644
--- a/tools/perf/util/top.h
+++ b/tools/perf/util/top.h
@@ -29,6 +29,7 @@ struct perf_top {
 	bool		   use_tui, use_stdio;
 	bool		   kptr_restrict_warned;
 	bool		   vmlinux_warned;
+	bool		   enomem_warned;
 	bool		   dump_symtab;
 	struct hist_entry  *sym_filter_entry;
 	struct perf_evsel  *sym_evsel;
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1288281 — [PATCH/RFC 15/16] perf top: Add --num-thread option

The --num-thread option is to set number of reader thread.  Default
value is 0 which will be converted to 1/4 of number of mmaps.

Signed-off-by: Namhyung Kim <namhyung@kernel.org>
---
 tools/perf/builtin-top.c | 49 +++++++++++++++++++++++++++++++++++-------------
 tools/perf/util/top.h    |  1 +
 2 files changed, 37 insertions(+), 13 deletions(-)

diff --git a/tools/perf/builtin-top.c b/tools/perf/builtin-top.c
index f3ab46b234b6..fc9715b046b3 100644
--- a/tools/perf/builtin-top.c
+++ b/tools/perf/builtin-top.c
@@ -838,10 +838,10 @@ struct collector_arg {
 
 static void collect_hists(struct perf_top *top, struct hists *hists)
 {
-	int i, k;
+	unsigned int i, k;
 	struct perf_evsel *evsel;
 
-	for (i = 0, k = 0; i < top->evlist->nr_mmaps; i++) {
+	for (i = 0, k = 0; i < top->nr_threads; i++) {
 		evlist__for_each(top->evlist, evsel) {
 			struct hists *src_hists = &hists[k++];
 			struct hists *dst_hists = evsel__hists(evsel);
@@ -900,6 +900,7 @@ static int hist_iter__top_callback(struct hist_entry_iter *iter,
 
 struct reader_arg {
 	int			idx;
+	int			nr_idx;
 	struct perf_top		*top;
 	struct hists		*hists;
 	struct perf_top_stats	stats;
@@ -993,7 +994,7 @@ static void perf_event__process_sample(struct reader_arg *rarg,
 	addr_location__put(&al);
 }
 
-static void perf_top__mmap_read(struct reader_arg *rarg)
+static void perf_top__mmap_read_idx(struct reader_arg *rarg, int idx)
 {
 	struct perf_sample sample;
 	struct perf_evsel *evsel;
@@ -1001,7 +1002,6 @@ static void perf_top__mmap_read(struct reader_arg *rarg)
 	struct perf_session *session = top->session;
 	union perf_event *event;
 	struct machine *machine;
-	int idx = rarg->idx;
 	u8 origin;
 	int ret;
 
@@ -1067,6 +1067,14 @@ static void perf_top__mmap_read(struct reader_arg *rarg)
 	}
 }
 
+static void perf_top__mmap_read(struct reader_arg *rarg)
+{
+	int i;
+
+	for (i = 0; i < rarg->nr_idx; i++)
+		perf_top__mmap_read_idx(rarg, rarg->idx + i);
+}
+
 static void *mmap_read_worker(void *arg)
 {
 	struct reader_arg *rarg = arg;
@@ -1160,7 +1168,8 @@ static int __cmd_top(struct perf_top *top)
 	struct reader_arg *rargs = NULL;
 	struct collector_arg carg;
 	int ret;
-	int i;
+	unsigned int i;
+	int idx, nr_idx, rem;
 
 	top->session = perf_session__new(NULL, false, NULL);
 	if (top->session == NULL)
@@ -1211,34 +1220,47 @@ static int __cmd_top(struct perf_top *top)
 	/* Wait for a minimal set of events before starting the snapshot */
 	perf_evlist__poll(top->evlist, 100);
 
+	if (top->nr_threads == 0)
+		top->nr_threads = top->evlist->nr_mmaps / 4 ?: 1;
+	if ((int)top->nr_threads > top->evlist->nr_mmaps)
+		top->nr_threads = top->evlist->nr_mmaps;
+
+	nr_idx = top->evlist->nr_mmaps / top->nr_threads;
+	rem = top->evlist->nr_mmaps % top->nr_threads;
+
 	ret = -1;
-	readers = calloc(sizeof(pthread_t), top->evlist->nr_mmaps);
+	readers = calloc(sizeof(pthread_t), top->nr_threads);
 	if (readers == NULL)
 		goto out_delete;
 
-	rargs = calloc(sizeof(*rargs), top->evlist->nr_mmaps);
+	rargs = calloc(sizeof(*rargs), top->nr_threads);
 	if (rargs == NULL)
 		goto out_free;
 
-	hists = calloc(sizeof(*hists), top->evlist->nr_mmaps * top->evlist->nr_entries);
+	hists = calloc(sizeof(*hists), top->nr_threads * top->evlist->nr_entries);
 	if (hists == NULL)
 		goto out_free;
 
-	for (i = 0; i < top->evlist->nr_mmaps * top->evlist->nr_entries; i++)
+	for (i = 0; i < top->nr_threads * top->evlist->nr_entries; i++)
 		__hists__init(&hists[i]);
 
-	for (i = 0; i < top->evlist->nr_mmaps; i++) {
+	for (i = 0, idx = 0; i < top->nr_threads; i++) {
 		struct reader_arg *rarg = &rargs[i];
 
-		rarg->idx = i;
 		rarg->top = top;
 		rarg->hists = &hists[i * top->evlist->nr_entries];
 
+		rarg->idx = idx;
+		rarg->nr_idx = nr_idx;
+		if (rem-- > 0)
+			rarg->nr_idx++;
+		idx += rarg->nr_idx;
+
 		perf_top__mmap_read(rarg);
 	}
 	collect_hists(top, hists);
 
-	for (i = 0; i < top->evlist->nr_mmaps; i++) {
+	for (i = 0; i < top->nr_threads; i++) {
 		if (pthread_create(&readers[i], NULL, mmap_read_worker, &rargs[i]))
 			goto out_join;
 	}
@@ -1259,7 +1281,7 @@ static int __cmd_top(struct perf_top *top)
 out_join:
 	pthread_join(ui_thread, NULL);
 	pthread_join(collector, NULL);
-	for (i = 0; i < top->evlist->nr_mmaps; i++) {
+	for (i = 0; i < top->nr_threads; i++) {
 		pthread_join(readers[i], NULL);
 	}
 
@@ -1458,6 +1480,7 @@ int cmd_top(int argc, const char **argv, const char *prefix __maybe_unused)
 	OPT_CALLBACK('j', "branch-filter", &opts->branch_stack,
 		     "branch filter mask", "branch stack filter modes",
 		     parse_branch_stack),
+	OPT_UINTEGER(0, "num-thread", &top.nr_threads, "number of thread to run"),
 	OPT_END()
 	};
 	const char * const top_usage[] = {
diff --git a/tools/perf/util/top.h b/tools/perf/util/top.h
index 55eb5aebae59..916ba36b0ac0 100644
--- a/tools/perf/util/top.h
+++ b/tools/perf/util/top.h
@@ -43,6 +43,7 @@ struct perf_top {
 	int		   sym_pcnt_filter;
 	const char	   *sym_filter;
 	float		   min_percent;
+	unsigned int	   nr_threads;
 };
 
 #define CONSOLE_CLEAR ""
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1288282 — [PATCH/RFC 09/16] perf tools: Update hist entry's hists pointer

When sample is processed using multi-thread, each sample is gathered on
each thread's hist tree and then merged into the real hist tree.  But
hist_entry->hists pointer was not updated so it could refer wrong hists
resulted in missing outputs.

Signed-off-by: Namhyung Kim <namhyung@kernel.org>
---
 tools/perf/util/hist.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/tools/perf/util/hist.c b/tools/perf/util/hist.c
index ea4f3ad978b0..a12e5022fe04 100644
--- a/tools/perf/util/hist.c
+++ b/tools/perf/util/hist.c
@@ -1008,6 +1008,14 @@ bool hists__collapse_insert_entry(struct hists *hists __maybe_unused,
 	}
 	hists->nr_entries++;
 
+	/*
+	 * If a hist entry is processed in multi-threaded environment,
+	 * it points to a dummy local hists which was used only for
+	 * intermidate processing.  So update it to a real one so that
+	 * it can find the correct info later.
+	 */
+	he->hists = hists;
+
 	rb_link_node(&he->rb_node_in, parent, p);
 	rb_insert_color(&he->rb_node_in, root);
 	return true;
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1288286

* Namhyung Kim <namhyung@kernel.org> wrote:

> Hello,
> 
> This patchset if an attempt to support multi-threading in perf top.
> In fact, perf top already run on two threads - a worker thread and a
> display thread.  However processing all samples with a single thread
> in a large machine can have scalability problems.
> 
> This patchset extends it to have multiple worker threads to process
> samples concurrently.  Users can control the number of threads using
>  --num-thread option.  And there's a collector thread for passing hist
> entries from worker threads to the display thread.

Could you please make the number of threads default to the number of CPUs?

Since perf top is doing one perf event per CPU anyway, that's a pretty natural 
model.

( I think 'perf record' should use per CPU threads as well to receive events, to 
  address the 'IO overload' problems with -g recording on larger CPU counts. )

Thanks,

	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1288362

On December 10, 2015 5:01:18 PM GMT+09:00, Ingo Molnar <mingo@kernel.org> wrote:
>
>* Namhyung Kim <namhyung@kernel.org> wrote:
>
>> Hello,
>> 
>> This patchset if an attempt to support multi-threading in perf top.
>> In fact, perf top already run on two threads - a worker thread and a
>> display thread.  However processing all samples with a single thread
>> in a large machine can have scalability problems.
>> 
>> This patchset extends it to have multiple worker threads to process
>> samples concurrently.  Users can control the number of threads using
>>  --num-thread option.  And there's a collector thread for passing
>hist
>> entries from worker threads to the display thread.
>
>Could you please make the number of threads default to the number of
>CPUs?
>
>Since perf top is doing one perf event per CPU anyway, that's a pretty
>natural 
>model.
>
>( I think 'perf record' should use per CPU threads as well to receive
>events, to 
>address the 'IO overload' problems with -g recording on larger CPU
>counts. )

IIRC David said that thread per cpu seems too much
especially on a large system (like ~1024 cpu).  I have no idea 
what's the reasonable default on the system, so I chose 1/4 
of map buffers (i.e. cpus for most cases). But I think I should
take non-system-wide mode into account too.

Thanks
Namhyung

Hi Ingo,
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1289301

* Namhyung Kim <namhyung@gmail.com> wrote:

> IIRC David said that thread per cpu seems too much especially on a large system 
> (like ~1024 cpu). [...]

Too much in what fashion? For recording I think it's the fastest, most natural 
model - anything else will create cache line bounces.

For perf report, I suspect you are right, it would depend on the actual possible 
parallelism - which with time would improve I suspect.

Thanks,

	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1289656

On 12/11/15 1:11 AM, Ingo Molnar wrote:
>
> * Namhyung Kim <namhyung@gmail.com> wrote:
>
>> IIRC David said that thread per cpu seems too much especially on a large system
>> (like ~1024 cpu). [...]
>
> Too much in what fashion? For recording I think it's the fastest, most natural
> model - anything else will create cache line bounces.

The intrusiveness of perf on the system under observation. I understand 
there are a lot of factors that go into it.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web