Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1693337 > unrolled thread

[PATCH 0/5] perf script python: Provide perf_sample dict to all handlers

Back to article view | Back to linux.kernel

Contents

  [PATCH 0/5] perf script python: Provide perf_sample dict to all handlers Arun Kalyanasundaram <arunkaly@google.com> - 2017-07-21 04:10 +0200
    [PATCH 5/5] perf script python: Generate hooks with additional argument Arun Kalyanasundaram <arunkaly@google.com> - 2017-07-21 04:10 +0200
    [PATCH 3/5] perf script python: Add sample_read to dict Arun Kalyanasundaram <arunkaly@google.com> - 2017-07-21 04:10 +0200
    [PATCH 4/5] perf script python: Add perf_sample dict to tracepoint handlers Arun Kalyanasundaram <arunkaly@google.com> - 2017-07-21 04:10 +0200
    Re: [PATCH 0/5] perf script python: Provide perf_sample dict to all  handlers Jiri Olsa <jolsa@redhat.com> - 2017-07-21 09:50 +0200
      Re: [PATCH 0/5] perf script python: Provide perf_sample dict to all  handlers Arnaldo Carvalho de Melo <acme@kernel.org> - 2017-07-21 18:30 +0200
        Re: [PATCH 0/5] perf script python: Provide perf_sample dict to all handlers Arun Kalyanasundaram <arunkaly@google.com> - 2017-07-21 19:00 +0200
          Re: [PATCH 0/5] perf script python: Provide perf_sample dict to all  handlers Arnaldo Carvalho de Melo <acme@kernel.org> - 2017-07-21 19:20 +0200

#1693337 — [PATCH 0/5] perf script python: Provide perf_sample dict to all handlers

The process_event python hook receives a dict with most perf_sample
entries.

Other handlers (e.g. trace_unhandled, python_process_tracepoint) predate
the introduction of this dict and do not receive it. This patch series
adds the dict to all handlers, aiming to unify the information passed to
them.

This change adds an additional argument to the affected handlers. To
keep backwards compatibility (and avoid unnecessary work), do not pass
the aforementioned dict if the number of arguments signals that handler
version predates this change.

In addition, provide time_enabled, time_running and counter value in the
perf_sample dict.

Initial Discussion: https://lkml.org/lkml/2017/7/1/108

Arun Kalyanasundaram (5):
  perf script python: Allocate memory only if handler exists
  perf script python: Refactor creation of perf sample dict
  perf script python: Add sample_read to dict
  perf script python: Add perf_sample dict to tracepoint handlers
  perf script python: Generate hooks with additional argument

 .../util/scripting-engines/trace-event-python.c    | 246 +++++++++++++++------
 1 file changed, 184 insertions(+), 62 deletions(-)

-- 
2.14.0.rc0.284.gd933b75aa4-goog

[toc] | [next] | [standalone]

#1693339 — [PATCH 5/5] perf script python: Generate hooks with additional argument

Modify the signature of tracepoint specific and trace_unhandled hooks to
add the perf_sample dict as a new argument.
Create a python helper function to print a dictionary.

Signed-off-by: Arun Kalyanasundaram <arunkaly@google.com>
---
 .../util/scripting-engines/trace-event-python.c    | 22 ++++++++++++++++++----
 1 file changed, 18 insertions(+), 4 deletions(-)

diff --git a/tools/perf/util/scripting-engines/trace-event-python.c b/tools/perf/util/scripting-engines/trace-event-python.c
index 938b39f6ad31..c7187f067d31 100644
--- a/tools/perf/util/scripting-engines/trace-event-python.c
+++ b/tools/perf/util/scripting-engines/trace-event-python.c
@@ -1367,6 +1367,12 @@ static int python_generate_script(struct pevent *pevent, const char *outfile)
 
 			fprintf(ofp, "%s", f->name);
 		}
+		if (not_first++)
+			fprintf(ofp, ", ");
+		if (++count % 5 == 0)
+			fprintf(ofp, "\n\t\t");
+		fprintf(ofp, "perf_sample_dict");
+
 		fprintf(ofp, "):\n");
 
 		fprintf(ofp, "\t\tprint_header(event_name, common_cpu, "
@@ -1436,6 +1442,9 @@ static int python_generate_script(struct pevent *pevent, const char *outfile)
 
 		fprintf(ofp, ")\n\n");
 
+		fprintf(ofp, "\t\tprint 'Sample: {'+"
+			"get_dict_as_string(perf_sample_dict['sample'], ', ')+'}'\n\n");
+
 		fprintf(ofp, "\t\tfor node in common_callchain:");
 		fprintf(ofp, "\n\t\t\tif 'sym' in node:");
 		fprintf(ofp, "\n\t\t\t\tprint \"\\t[%%x] %%s\" %% (node['ip'], node['sym']['name'])");
@@ -1446,15 +1455,20 @@ static int python_generate_script(struct pevent *pevent, const char *outfile)
 	}
 
 	fprintf(ofp, "def trace_unhandled(event_name, context, "
-		"event_fields_dict):\n");
+		"event_fields_dict, perf_sample_dict):\n");
 
-	fprintf(ofp, "\t\tprint ' '.join(['%%s=%%s'%%(k,str(v))"
-		"for k,v in sorted(event_fields_dict.items())])\n\n");
+	fprintf(ofp, "\t\tprint get_dict_as_string(event_fields_dict)\n");
+	fprintf(ofp, "\t\tprint 'Sample: {'+"
+		"get_dict_as_string(perf_sample_dict['sample'], ', ')+'}'\n\n");
 
 	fprintf(ofp, "def print_header("
 		"event_name, cpu, secs, nsecs, pid, comm):\n"
 		"\tprint \"%%-20s %%5u %%05u.%%09u %%8u %%-20s \" %% \\\n\t"
-		"(event_name, cpu, secs, nsecs, pid, comm),\n");
+		"(event_name, cpu, secs, nsecs, pid, comm),\n\n");
+
+	fprintf(ofp, "def get_dict_as_string(a_dict, delimiter=' '):\n"
+		"\treturn delimiter.join"
+		"(['%%s=%%s'%%(k,str(v))for k,v in sorted(a_dict.items())])\n");
 
 	fclose(ofp);
 
-- 
2.14.0.rc0.284.gd933b75aa4-goog

[toc] | [prev] | [next] | [standalone]

#1693340 — [PATCH 3/5] perf script python: Add sample_read to dict

Provide time_enabled, time_running and counter value in the perf_sample
dict.

Signed-off-by: Arun Kalyanasundaram <arunkaly@google.com>
---
 .../util/scripting-engines/trace-event-python.c    | 51 ++++++++++++++++++++++
 1 file changed, 51 insertions(+)

diff --git a/tools/perf/util/scripting-engines/trace-event-python.c b/tools/perf/util/scripting-engines/trace-event-python.c
index 69d1b6db96f6..55a45784c910 100644
--- a/tools/perf/util/scripting-engines/trace-event-python.c
+++ b/tools/perf/util/scripting-engines/trace-event-python.c
@@ -391,6 +391,56 @@ static PyObject *python_process_callchain(struct perf_sample *sample,
 	return pylist;
 }
 
+static PyObject *get_sample_value_as_tuple(struct sample_read_value *value)
+{
+	PyObject *t;
+
+	t = PyTuple_New(2);
+	if (!t)
+		Py_FatalError("couldn't create Python tuple");
+	PyTuple_SetItem(t, 0, PyLong_FromUnsignedLongLong(value->id));
+	PyTuple_SetItem(t, 1, PyLong_FromUnsignedLongLong(value->value));
+	return t;
+}
+
+static void set_sample_read_in_dict(PyObject *dict_sample,
+					 struct perf_sample *sample,
+					 struct perf_evsel *evsel)
+{
+	u64 read_format = evsel->attr.read_format;
+	PyObject *values;
+	unsigned int i;
+
+	if (read_format & PERF_FORMAT_TOTAL_TIME_ENABLED) {
+		pydict_set_item_string_decref(dict_sample, "time_enabled",
+			PyLong_FromUnsignedLongLong(sample->read.time_enabled));
+	}
+
+	if (read_format & PERF_FORMAT_TOTAL_TIME_RUNNING) {
+		pydict_set_item_string_decref(dict_sample, "time_running",
+			PyLong_FromUnsignedLongLong(sample->read.time_running));
+	}
+
+	if (read_format & PERF_FORMAT_GROUP)
+		values = PyList_New(sample->read.group.nr);
+	else
+		values = PyList_New(1);
+
+	if (!values)
+		Py_FatalError("couldn't create Python list");
+
+	if (read_format & PERF_FORMAT_GROUP) {
+		for (i = 0; i < sample->read.group.nr; i++) {
+			PyObject *t = get_sample_value_as_tuple(&sample->read.group.values[i]);
+			PyList_SET_ITEM(values, i, t);
+		}
+	} else {
+		PyObject *t = get_sample_value_as_tuple(&sample->read.one);
+		PyList_SET_ITEM(values, 0, t);
+	}
+	pydict_set_item_string_decref(dict_sample, "values", values);
+}
+
 static PyObject *get_perf_sample_dict(struct perf_sample *sample,
 					 struct perf_evsel *evsel,
 					 struct addr_location *al,
@@ -422,6 +472,7 @@ static PyObject *get_perf_sample_dict(struct perf_sample *sample,
 			PyLong_FromUnsignedLongLong(sample->time));
 	pydict_set_item_string_decref(dict_sample, "period",
 			PyLong_FromUnsignedLongLong(sample->period));
+	set_sample_read_in_dict(dict_sample, sample, evsel);
 	pydict_set_item_string_decref(dict, "sample", dict_sample);
 
 	pydict_set_item_string_decref(dict, "raw_buf", PyString_FromStringAndSize(
-- 
2.14.0.rc0.284.gd933b75aa4-goog

[toc] | [prev] | [next] | [standalone]

#1693343 — [PATCH 4/5] perf script python: Add perf_sample dict to tracepoint handlers

The process_event python hook receives a dict with all perf_sample
entries, but the tracepoint specific and trace_unhandled hooks predate
the introduction of this dict, and do not receive it.

Add the aforementioned dict as an additional argument to the affected
handlers. To keep backwards compatibility (and avoid unnecessary work),
do not pass the dict if the number of arguments signals that handler
version predates this change.

Signed-off-by: Arun Kalyanasundaram <arunkaly@google.com>
---
 .../util/scripting-engines/trace-event-python.c    | 41 +++++++++++++++++++++-
 1 file changed, 40 insertions(+), 1 deletion(-)

diff --git a/tools/perf/util/scripting-engines/trace-event-python.c b/tools/perf/util/scripting-engines/trace-event-python.c
index 55a45784c910..938b39f6ad31 100644
--- a/tools/perf/util/scripting-engines/trace-event-python.c
+++ b/tools/perf/util/scripting-engines/trace-event-python.c
@@ -116,6 +116,34 @@ static PyObject *get_handler(const char *handler_name)
 	return handler;
 }
 
+static int get_argument_count(PyObject *handler)
+{
+	int arg_count = 0;
+
+	/*
+	 * The attribute for the code object is func_code in Python 2,
+	 * whereas it is __code__ in Python 3.0+.
+	 */
+	PyObject *code_obj = PyObject_GetAttrString(handler,
+		"func_code");
+	if (PyErr_Occurred()) {
+		PyErr_Clear();
+		code_obj = PyObject_GetAttrString(handler,
+			"__code__");
+	}
+	PyErr_Clear();
+	if (code_obj) {
+		PyObject *arg_count_obj = PyObject_GetAttrString(code_obj,
+			"co_argcount");
+		if (arg_count_obj) {
+			arg_count = (int) PyInt_AsLong(arg_count_obj);
+			Py_DECREF(arg_count_obj);
+		}
+		Py_DECREF(code_obj);
+	}
+	return arg_count;
+}
+
 static void call_object(PyObject *handler, PyObject *args, const char *die_msg)
 {
 	PyObject *retval;
@@ -499,7 +527,7 @@ static void python_process_tracepoint(struct perf_sample *sample,
 {
 	struct event_format *event = evsel->tp_format;
 	PyObject *handler, *context, *t, *obj = NULL, *callchain;
-	PyObject *dict = NULL;
+	PyObject *dict = NULL, *all_entries_dict = NULL;
 	static char handler_name[256];
 	struct format_field *field;
 	unsigned long s, ns;
@@ -552,6 +580,8 @@ static void python_process_tracepoint(struct perf_sample *sample,
 
 	/* ip unwinding */
 	callchain = python_process_callchain(sample, evsel, al);
+	/* Need an additional reference for the perf_sample dict */
+	Py_INCREF(callchain);
 
 	if (!dict) {
 		PyTuple_SetItem(t, n++, PyInt_FromLong(cpu));
@@ -602,6 +632,14 @@ static void python_process_tracepoint(struct perf_sample *sample,
 	if (dict)
 		PyTuple_SetItem(t, n++, dict);
 
+	if (get_argument_count(handler) == (int) n + 1) {
+		all_entries_dict = get_perf_sample_dict(sample, evsel, al,
+			callchain);
+		PyTuple_SetItem(t, n++,	all_entries_dict);
+	} else {
+		Py_DECREF(callchain);
+	}
+
 	if (_PyTuple_Resize(&t, n) == -1)
 		Py_FatalError("error resizing Python tuple");
 
@@ -612,6 +650,7 @@ static void python_process_tracepoint(struct perf_sample *sample,
 		Py_DECREF(dict);
 	}
 
+	Py_XDECREF(all_entries_dict);
 	Py_DECREF(t);
 }
 
-- 
2.14.0.rc0.284.gd933b75aa4-goog

[toc] | [prev] | [next] | [standalone]

#1693457 — Re: [PATCH 0/5] perf script python: Provide perf_sample dict to all handlers

On Thu, Jul 20, 2017 at 07:01:13PM -0700, Arun Kalyanasundaram wrote:
> The process_event python hook receives a dict with most perf_sample
> entries.

hi,
was there any change to the rfc post?

thanks,
jirka

> 
> Other handlers (e.g. trace_unhandled, python_process_tracepoint) predate
> the introduction of this dict and do not receive it. This patch series
> adds the dict to all handlers, aiming to unify the information passed to
> them.
> 
> This change adds an additional argument to the affected handlers. To
> keep backwards compatibility (and avoid unnecessary work), do not pass
> the aforementioned dict if the number of arguments signals that handler
> version predates this change.
> 
> In addition, provide time_enabled, time_running and counter value in the
> perf_sample dict.
> 
> Initial Discussion: https://lkml.org/lkml/2017/7/1/108
> 
> Arun Kalyanasundaram (5):
>   perf script python: Allocate memory only if handler exists
>   perf script python: Refactor creation of perf sample dict
>   perf script python: Add sample_read to dict
>   perf script python: Add perf_sample dict to tracepoint handlers
>   perf script python: Generate hooks with additional argument
> 
>  .../util/scripting-engines/trace-event-python.c    | 246 +++++++++++++++------
>  1 file changed, 184 insertions(+), 62 deletions(-)
> 
> -- 
> 2.14.0.rc0.284.gd933b75aa4-goog
> 

[toc] | [prev] | [next] | [standalone]

#1693833 — Re: [PATCH 0/5] perf script python: Provide perf_sample dict to all handlers

Em Fri, Jul 21, 2017 at 09:46:39AM +0200, Jiri Olsa escreveu:
> On Thu, Jul 20, 2017 at 07:01:13PM -0700, Arun Kalyanasundaram wrote:
> > The process_event python hook receives a dict with most perf_sample
> > entries.
> 
> hi,
> was there any change to the rfc post?

I was wondering that, please stick a v1, v2, etc to the subject, and
state the changes across versions, to make things clear.

- Arnaldo
 
> thanks,
> jirka
> 
> > 
> > Other handlers (e.g. trace_unhandled, python_process_tracepoint) predate
> > the introduction of this dict and do not receive it. This patch series
> > adds the dict to all handlers, aiming to unify the information passed to
> > them.
> > 
> > This change adds an additional argument to the affected handlers. To
> > keep backwards compatibility (and avoid unnecessary work), do not pass
> > the aforementioned dict if the number of arguments signals that handler
> > version predates this change.
> > 
> > In addition, provide time_enabled, time_running and counter value in the
> > perf_sample dict.
> > 
> > Initial Discussion: https://lkml.org/lkml/2017/7/1/108
> > 
> > Arun Kalyanasundaram (5):
> >   perf script python: Allocate memory only if handler exists
> >   perf script python: Refactor creation of perf sample dict
> >   perf script python: Add sample_read to dict
> >   perf script python: Add perf_sample dict to tracepoint handlers
> >   perf script python: Generate hooks with additional argument
> > 
> >  .../util/scripting-engines/trace-event-python.c    | 246 +++++++++++++++------
> >  1 file changed, 184 insertions(+), 62 deletions(-)
> > 
> > -- 
> > 2.14.0.rc0.284.gd933b75aa4-goog
> > 

[toc] | [prev] | [next] | [standalone]

#1693865

My apologies. Yes, I did make a couple of changes to the patch. I was
not sure if I should be sending a v2 since the previous one was a rfc.
Please ignore this patch, I will resend this highlighting the new
changes made.

On Fri, Jul 21, 2017 at 9:28 AM, Arnaldo Carvalho de Melo
<acme@kernel.org> wrote:
> Em Fri, Jul 21, 2017 at 09:46:39AM +0200, Jiri Olsa escreveu:
>> On Thu, Jul 20, 2017 at 07:01:13PM -0700, Arun Kalyanasundaram wrote:
>> > The process_event python hook receives a dict with most perf_sample
>> > entries.
>>
>> hi,
>> was there any change to the rfc post?
>
> I was wondering that, please stick a v1, v2, etc to the subject, and
> state the changes across versions, to make things clear.
>
> - Arnaldo
>
>> thanks,
>> jirka
>>
>> >
>> > Other handlers (e.g. trace_unhandled, python_process_tracepoint) predate
>> > the introduction of this dict and do not receive it. This patch series
>> > adds the dict to all handlers, aiming to unify the information passed to
>> > them.
>> >
>> > This change adds an additional argument to the affected handlers. To
>> > keep backwards compatibility (and avoid unnecessary work), do not pass
>> > the aforementioned dict if the number of arguments signals that handler
>> > version predates this change.
>> >
>> > In addition, provide time_enabled, time_running and counter value in the
>> > perf_sample dict.
>> >
>> > Initial Discussion: https://lkml.org/lkml/2017/7/1/108
>> >
>> > Arun Kalyanasundaram (5):
>> >   perf script python: Allocate memory only if handler exists
>> >   perf script python: Refactor creation of perf sample dict
>> >   perf script python: Add sample_read to dict
>> >   perf script python: Add perf_sample dict to tracepoint handlers
>> >   perf script python: Generate hooks with additional argument
>> >
>> >  .../util/scripting-engines/trace-event-python.c    | 246 +++++++++++++++------
>> >  1 file changed, 184 insertions(+), 62 deletions(-)
>> >
>> > --
>> > 2.14.0.rc0.284.gd933b75aa4-goog
>> >

[toc] | [prev] | [next] | [standalone]

#1693873 — Re: [PATCH 0/5] perf script python: Provide perf_sample dict to all handlers

Em Fri, Jul 21, 2017 at 09:51:31AM -0700, Arun Kalyanasundaram escreveu:
> My apologies. Yes, I did make a couple of changes to the patch. I was
> not sure if I should be sending a v2 since the previous one was a rfc.
> Please ignore this patch, I will resend this highlighting the new
> changes made.

Thanks a lot! It takes some time to get the whole process going
smoothly, but we'll get there :-)

- Arnaldo
 
> On Fri, Jul 21, 2017 at 9:28 AM, Arnaldo Carvalho de Melo
> <acme@kernel.org> wrote:
> > Em Fri, Jul 21, 2017 at 09:46:39AM +0200, Jiri Olsa escreveu:
> >> On Thu, Jul 20, 2017 at 07:01:13PM -0700, Arun Kalyanasundaram wrote:
> >> > The process_event python hook receives a dict with most perf_sample
> >> > entries.
> >>
> >> hi,
> >> was there any change to the rfc post?
> >
> > I was wondering that, please stick a v1, v2, etc to the subject, and
> > state the changes across versions, to make things clear.
> >
> > - Arnaldo
> >
> >> thanks,
> >> jirka
> >>
> >> >
> >> > Other handlers (e.g. trace_unhandled, python_process_tracepoint) predate
> >> > the introduction of this dict and do not receive it. This patch series
> >> > adds the dict to all handlers, aiming to unify the information passed to
> >> > them.
> >> >
> >> > This change adds an additional argument to the affected handlers. To
> >> > keep backwards compatibility (and avoid unnecessary work), do not pass
> >> > the aforementioned dict if the number of arguments signals that handler
> >> > version predates this change.
> >> >
> >> > In addition, provide time_enabled, time_running and counter value in the
> >> > perf_sample dict.
> >> >
> >> > Initial Discussion: https://lkml.org/lkml/2017/7/1/108
> >> >
> >> > Arun Kalyanasundaram (5):
> >> >   perf script python: Allocate memory only if handler exists
> >> >   perf script python: Refactor creation of perf sample dict
> >> >   perf script python: Add sample_read to dict
> >> >   perf script python: Add perf_sample dict to tracepoint handlers
> >> >   perf script python: Generate hooks with additional argument
> >> >
> >> >  .../util/scripting-engines/trace-event-python.c    | 246 +++++++++++++++------
> >> >  1 file changed, 184 insertions(+), 62 deletions(-)
> >> >
> >> > --
> >> > 2.14.0.rc0.284.gd933b75aa4-goog
> >> >

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web