Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1215160 > unrolled thread

[PATCH 32/32] bpf: Introduce function for outputing data to perf event

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  [PATCH 32/32] bpf: Introduce function for outputing data to perf event Wang Nan <wangnan0@huawei.com> - 2015-08-28 09:10 +0200
    Re: [PATCH 32/32] bpf: Introduce function for outputing data to perf  event Alexei Starovoitov <ast@plumgrid.com> - 2015-08-29 02:50 +0200
      Re: [PATCH 32/32] bpf: Introduce function for outputing data to perf  event "Wangnan (F)" <wangnan0@huawei.com> - 2015-08-29 03:30 +0200
        Re: [PATCH 32/32] bpf: Introduce function for outputing data to perf  event Alexei Starovoitov <ast@plumgrid.com> - 2015-08-29 03:40 +0200
          Re: [PATCH 32/32] bpf: Introduce function for outputing data to perf  event "Wangnan (F)" <wangnan0@huawei.com> - 2015-08-29 04:20 +0200
            Re: [PATCH 32/32] bpf: Introduce function for outputing data to perf  event Alexei Starovoitov <ast@plumgrid.com> - 2015-08-29 04:30 +0200
              Re: [PATCH 32/32] bpf: Introduce function for outputing data to perf  event "Wangnan (F)" <wangnan0@huawei.com> - 2015-08-29 04:40 +0200
                Re: [PATCH 32/32] bpf: Introduce function for outputing data to perf  event Alexei Starovoitov <ast@plumgrid.com> - 2015-08-29 05:00 +0200
                  Re: [PATCH 32/32] bpf: Introduce function for outputing data to perf  event "Wangnan (F)" <wangnan0@huawei.com> - 2015-08-29 05:00 +0200

#1215160 — [PATCH 32/32] bpf: Introduce function for outputing data to perf event

From: He Kuang <hekuang@huawei.com>

There're scenarios that we need an eBPF program to record not only
kprobe point args, but also the PMU counters, time latencies or the
number of cache misses between two probe points and other information
when the probe point is entered.

This patch adds a new trace event to establish infrastruction for bpf to
output data to perf. Userspace perf tools can detect and use this event
as using the existing tracepoint events.

New bpf trace event entry in debugfs:

     /sys/kernel/debug/tracing/events/bpf/bpf_output_data

Userspace perf tools detect the new tracepoint event as:

     bpf:bpf_output_data                          [Tracepoint event]

Data in ring-buffer of perf events added to this event will be polled
out, sample types and other attributes can be adjusted to those events
directly without touching the original kprobe events.

The bpf helper function gives eBPF program ability to output data as
perf sample event. This helper simple call the new trace event and
userspace perf tools can record the BPF ftrace event to collect those
records.

Signed-off-by: He Kuang <hekuang@huawei.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: pi3orama@163.com
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/1437448130-134621-3-git-send-email-hekuang@huawei.com
---
 include/trace/events/bpf.h | 30 ++++++++++++++++++++++++++++++
 include/uapi/linux/bpf.h   |  7 +++++++
 kernel/trace/bpf_trace.c   | 23 +++++++++++++++++++++++
 samples/bpf/bpf_helpers.h  |  2 ++
 4 files changed, 62 insertions(+)
 create mode 100644 include/trace/events/bpf.h

diff --git a/include/trace/events/bpf.h b/include/trace/events/bpf.h
new file mode 100644
index 0000000..6b739b8
--- /dev/null
+++ b/include/trace/events/bpf.h
@@ -0,0 +1,30 @@
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM bpf
+
+#if !defined(_TRACE_BPF_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_BPF_H
+
+#include <linux/tracepoint.h>
+
+TRACE_EVENT(bpf_output_data,
+
+	TP_PROTO(u64 *src, int size),
+
+	TP_ARGS(src, size),
+
+	TP_STRUCT__entry(
+		__dynamic_array(u8,		buf,		size)
+	),
+
+	TP_fast_assign(
+		memcpy(__get_dynamic_array(buf), src, size);
+	),
+
+	TP_printk("%s", __print_hex(__get_dynamic_array(buf),
+				    __get_dynamic_array_len(buf)))
+);
+
+#endif /* _TRACE_BPF_H */
+
+/* This part must be outside protection */
+#include <trace/define_trace.h>
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 29ef6f9..5068ab1 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -249,6 +249,13 @@ enum bpf_func_id {
 	 * Return: 0 on success
 	 */
 	BPF_FUNC_get_current_comm,
+
+	/**
+	 * int bpf_output_trace_data(void *src, int size)
+	 * Return: 0 on success
+	 */
+	BPF_FUNC_output_trace_data,
+
 	__BPF_FUNC_MAX_ID,
 };
 
diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
index 88a041a..219f670 100644
--- a/kernel/trace/bpf_trace.c
+++ b/kernel/trace/bpf_trace.c
@@ -11,7 +11,10 @@
 #include <linux/filter.h>
 #include <linux/uaccess.h>
 #include <linux/ctype.h>
+
 #include "trace.h"
+#define CREATE_TRACE_POINTS
+#include <trace/events/bpf.h>
 
 static DEFINE_PER_CPU(int, bpf_prog_active);
 
@@ -79,6 +82,24 @@ static const struct bpf_func_proto bpf_probe_read_proto = {
 	.arg3_type	= ARG_ANYTHING,
 };
 
+static u64 bpf_output_trace_data(u64 r1, u64 r2, u64 r3, u64 r4, u64 r5)
+{
+	void *src = (void *) (long) r1;
+	int size = (int) r2;
+
+	trace_bpf_output_data(src, size);
+
+	return 0;
+}
+
+static const struct bpf_func_proto bpf_output_trace_data_proto = {
+	.func		= bpf_output_trace_data,
+	.gpl_only	= true,
+	.ret_type	= RET_INTEGER,
+	.arg1_type	= ARG_PTR_TO_STACK,
+	.arg2_type	= ARG_CONST_STACK_SIZE,
+};
+
 /*
  * limited trace_printk()
  * only %d %u %x %ld %lu %lx %lld %llu %llx %p conversion specifiers allowed
@@ -169,6 +190,8 @@ static const struct bpf_func_proto *kprobe_prog_func_proto(enum bpf_func_id func
 		return &bpf_map_delete_elem_proto;
 	case BPF_FUNC_probe_read:
 		return &bpf_probe_read_proto;
+	case BPF_FUNC_output_trace_data:
+		return &bpf_output_trace_data_proto;
 	case BPF_FUNC_ktime_get_ns:
 		return &bpf_ktime_get_ns_proto;
 	case BPF_FUNC_tail_call:
diff --git a/samples/bpf/bpf_helpers.h b/samples/bpf/bpf_helpers.h
index bdf1c16..0aeaebe 100644
--- a/samples/bpf/bpf_helpers.h
+++ b/samples/bpf/bpf_helpers.h
@@ -59,5 +59,7 @@ static int (*bpf_l3_csum_replace)(void *ctx, int off, int from, int to, int flag
 	(void *) BPF_FUNC_l3_csum_replace;
 static int (*bpf_l4_csum_replace)(void *ctx, int off, int from, int to, int flags) =
 	(void *) BPF_FUNC_l4_csum_replace;
+static int (*bpf_output_trace_data)(void *src, int size) =
+	(void *) BPF_FUNC_output_trace_data;
 
 #endif
-- 
2.1.0

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1215627 — Re: [PATCH 32/32] bpf: Introduce function for outputing data to perf event

On 8/28/15 12:06 AM, Wang Nan wrote:
> his patch adds a new trace event to establish infrastruction for bpf to
> output data to perf. Userspace perf tools can detect and use this event
> as using the existing tracepoint events.
>
> New bpf trace event entry in debugfs:
>
>       /sys/kernel/debug/tracing/events/bpf/bpf_output_data
>
> Userspace perf tools detect the new tracepoint event as:
>
>       bpf:bpf_output_data                          [Tracepoint event]
>
> Data in ring-buffer of perf events added to this event will be polled
> out, sample types and other attributes can be adjusted to those events
> directly without touching the original kprobe events.

Wang,
I have 2nd thoughts on this.
I've played with it, but global bpf:bpf_output_data event is limiting.
I'd like to use this bpf_output_trace_data() helper for tcp estats
gathering, but global collector will prevent other similar bpf programs
running in parallel.
So as a concept I think it's very useful, but we need a way to select
which ring-buffer to output data to.
proposal A:
Can we use ftrace:instances concept and make bpf_output_trace_data()
into that particular trace_pipe ?
proposal B:
bpf_perf_event_read() model is using nice concept of an array of
perf_events. Can we perf_event_open a 'new' event that can be mmaped
in user space and bpf_output_trace_data(idx, buf, buf_size) into it.
Where 'idx' will be an index of FD from perf_even_open of such
new event?

Thanks!

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1215639 — Re: [PATCH 32/32] bpf: Introduce function for outputing data to perf event

On 2015/8/29 8:45, Alexei Starovoitov wrote:
> On 8/28/15 12:06 AM, Wang Nan wrote:
>> his patch adds a new trace event to establish infrastruction for bpf to
>> output data to perf. Userspace perf tools can detect and use this event
>> as using the existing tracepoint events.
>>
>> New bpf trace event entry in debugfs:
>>
>>       /sys/kernel/debug/tracing/events/bpf/bpf_output_data
>>
>> Userspace perf tools detect the new tracepoint event as:
>>
>>       bpf:bpf_output_data                          [Tracepoint event]
>>
>> Data in ring-buffer of perf events added to this event will be polled
>> out, sample types and other attributes can be adjusted to those events
>> directly without touching the original kprobe events.
>
> Wang,
> I have 2nd thoughts on this.
> I've played with it, but global bpf:bpf_output_data event is limiting.
> I'd like to use this bpf_output_trace_data() helper for tcp estats
> gathering, but global collector will prevent other similar bpf programs
> running in parallel.

So current model work for you but the problem is all output goes into one
place, which prevents similar BPF programs run in parallel because the
reveicer is unable to tell what message is generated by who. So actually
you want a publish-and-subscribe model, subscriber get messages from only
the publisher it interested in. Am I understand your problem correctly?

> So as a concept I think it's very useful, but we need a way to select
> which ring-buffer to output data to.
> proposal A:
> Can we use ftrace:instances concept and make bpf_output_trace_data()
> into that particular trace_pipe ?
> proposal B:
> bpf_perf_event_read() model is using nice concept of an array of
> perf_events. Can we perf_event_open a 'new' event that can be mmaped
> in user space and bpf_output_trace_data(idx, buf, buf_size) into it.
> Where 'idx' will be an index of FD from perf_even_open of such
> new event?
>

I've also thinking about adding the extra id parameter in 
bpf_output_trace_data()
but it is for encoding the type of output data, which is totally different
from what you want.

For me, I use bpf_output_trace_data() to output information like PMU count
value. Perf is the only receiver, so global collector is perfect. Could you
please describe your usecase in more detail?

Thank you for using that feature!

> Thanks!
>


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1215641 — Re: [PATCH 32/32] bpf: Introduce function for outputing data to perf event

On 8/28/15 6:19 PM, Wangnan (F) wrote:
> For me, I use bpf_output_trace_data() to output information like PMU count
> value. Perf is the only receiver, so global collector is perfect. Could you
> please describe your usecase in more detail?

there is a special receiver in user space that only wants the data from
the bpf program that it loaded. It shouldn't conflict with any other
processes. Like when it's running, I still should be able to use perf
for other performance analysis. There is no way to share single
bpf:bpf_output_data event, since these user processes are completely
independent.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1215650 — Re: [PATCH 32/32] bpf: Introduce function for outputing data to perf event

On 2015/8/29 9:34, Alexei Starovoitov wrote:
> On 8/28/15 6:19 PM, Wangnan (F) wrote:
>> For me, I use bpf_output_trace_data() to output information like PMU 
>> count
>> value. Perf is the only receiver, so global collector is perfect. 
>> Could you
>> please describe your usecase in more detail?
>
> there is a special receiver in user space that only wants the data from
> the bpf program that it loaded. It shouldn't conflict with any other
> processes. Like when it's running, I still should be able to use perf
> for other performance analysis. There is no way to share single
> bpf:bpf_output_data event, since these user processes are completely
> independent.
>
I'd like to see whether it is possible to create dynamic tracepoints so
different receivers can listen on different tracepoints. For my side, maybe
I can encode format information into the new tracepoints so don't need
those LLVM patches.

For example:

# echo 'dynamic_tracepoint:mytracepoint <encode its format>' >> 
/sys/kernel/debug/tracing/dynamic_trace_events
# perf list
   ...
   dynamic_tracepoint:mytracepoint
   ...

In perf side we can encode the creation of dynamic tracepoint into 
bpf-loader
like what we currectly do for probing the kprobes.

This way reqires us to create a fresh new event source, in parallel with 
tracepoint. I'm not sure
how much work it needs. What do you think?

Thank you.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1215653 — Re: [PATCH 32/32] bpf: Introduce function for outputing data to perf event

On 8/28/15 7:15 PM, Wangnan (F) wrote:
> I'd like to see whether it is possible to create dynamic tracepoints so
> different receivers can listen on different tracepoints.

see my proposal A. I think ftrace instances might work for this.

I'm not sure about 'format' part though. Kernel side shouldn't be
aware of it. It's only the contract between bpf program and user process
that deals with it.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1215657 — Re: [PATCH 32/32] bpf: Introduce function for outputing data to perf event

On 2015/8/29 10:22, Alexei Starovoitov wrote:
> On 8/28/15 7:15 PM, Wangnan (F) wrote:
>> I'd like to see whether it is possible to create dynamic tracepoints so
>> different receivers can listen on different tracepoints.
>
> see my proposal A. I think ftrace instances might work for this.
>
> I'm not sure about 'format' part though. Kernel side shouldn't be
> aware of it. It's only the contract between bpf program and user process
> that deals with it.
>
It is an option. Let's keep an open mind now :)

For current patch 32/32, I think it is useful enough for some simple cases,
and we have already start using it internally. What about keep it as what it
is now and create a independent method for your usecase?

Thank you.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1215660 — Re: [PATCH 32/32] bpf: Introduce function for outputing data to perf event

On 8/28/15 7:36 PM, Wangnan (F) wrote:
> For current patch 32/32, I think it is useful enough for some simple cases,
> and we have already start using it internally. What about keep it as
> what it
> is now and create a independent method for your usecase?

well, though the patch is small and contained, I think we can do better
and define more generic helper. I believe Namhyung back in July had
the same concern.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1215661 — Re: [PATCH 32/32] bpf: Introduce function for outputing data to perf event

On 2015/8/29 10:49, Alexei Starovoitov wrote:
> On 8/28/15 7:36 PM, Wangnan (F) wrote:
>> For current patch 32/32, I think it is useful enough for some simple 
>> cases,
>> and we have already start using it internally. What about keep it as
>> what it
>> is now and create a independent method for your usecase?
>
> well, though the patch is small and contained, I think we can do better
> and define more generic helper. I believe Namhyung back in July had
> the same concern.
>
OK. I'll drop this one in my next pull request.

Thank you.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web