Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1605110 > unrolled thread

perf: Improve support for uncore JSON event lists

Started byAndi Kleen <andi@firstfloor.org>
First post2017-03-20 21:20 +0100
Last post2017-03-21 21:00 +0100
Articles 8 on this page of 28 — 5 participants

Back to article view | Back to linux.kernel

Contents

  perf: Improve support for uncore JSON event lists Andi Kleen <andi@firstfloor.org> - 2017-03-20 21:20 +0100
    [PATCH 12/13] perf, tools: Add support for MetricName JSON attribute Andi Kleen <andi@firstfloor.org> - 2017-03-20 21:20 +0100
      [tip:perf/core] perf pmu: Add support for MetricName JSON attribute tip-bot for Andi Kleen <tipbot@zytor.com> - 2017-03-24 20:00 +0100
    [PATCH 03/13] perf, tools, stat: Handle partially bad results with merging Andi Kleen <andi@firstfloor.org> - 2017-03-20 21:20 +0100
      [tip:perf/core] perf stat: Handle partially bad results with  merging tip-bot for Andi Kleen <tipbot@zytor.com> - 2017-03-24 19:50 +0100
    [PATCH 04/13] perf, tools: Factor out PMU matching in parser Andi Kleen <andi@firstfloor.org> - 2017-03-20 21:20 +0100
      [tip:perf/core] perf tools: Factor out PMU matching in parser tip-bot for Andi Kleen <tipbot@zytor.com> - 2017-03-24 20:00 +0100
    [PATCH 02/13] perf, tools, stat: Collapse identically named events Andi Kleen <andi@firstfloor.org> - 2017-03-20 21:30 +0100
      [tip:perf/core] perf stat: Collapse identically named events tip-bot for Andi Kleen <tipbot@zytor.com> - 2017-03-24 20:00 +0100
    [PATCH 01/13] perf, tools, stat: Factor out callback for collecting event values Andi Kleen <andi@firstfloor.org> - 2017-03-20 21:30 +0100
      [tip:perf/core] perf stat: Factor out callback for collecting event  values tip-bot for Andi Kleen <tipbot@zytor.com> - 2017-03-24 19:50 +0100
    [PATCH 06/13] perf, tools: Special case uncore_ prefix Andi Kleen <andi@firstfloor.org> - 2017-03-20 21:30 +0100
      [tip:perf/core] perf pmu: Special case uncore_ prefix tip-bot for Andi Kleen <tipbot@zytor.com> - 2017-03-24 20:00 +0100
    [PATCH 10/13] perf, tools, stat: Output JSON MetricExpr metric Andi Kleen <andi@firstfloor.org> - 2017-03-20 21:30 +0100
      Re: [PATCH 10/13] perf, tools, stat: Output JSON MetricExpr metric Jiri Olsa <jolsa@redhat.com> - 2017-03-21 15:50 +0100
        Re: [PATCH 10/13] perf, tools, stat: Output JSON MetricExpr metric Andi Kleen <andi@firstfloor.org> - 2017-03-21 16:50 +0100
          Re: [PATCH 10/13] perf, tools, stat: Output JSON MetricExpr metric Arnaldo Carvalho de Melo <acme@kernel.org> - 2017-03-21 20:50 +0100
            Re: [PATCH 10/13] perf, tools, stat: Output JSON MetricExpr metric Andi Kleen <ak@linux.intel.com> - 2017-03-21 21:00 +0100
      [tip:perf/core] perf stat: Output JSON MetricExpr metric tip-bot for Andi Kleen <tipbot@zytor.com> - 2017-03-24 20:00 +0100
    [PATCH 13/13] perf, tools, list: Move extra details printing to new option Andi Kleen <andi@firstfloor.org> - 2017-03-20 21:30 +0100
      [tip:perf/core] perf list: Move extra details printing to new  option tip-bot for Andi Kleen <tipbot@zytor.com> - 2017-03-24 20:00 +0100
    [PATCH 07/13] perf, tools: Add a simple expression parser for JSON Andi Kleen <andi@firstfloor.org> - 2017-03-20 21:30 +0100
      Re: [PATCH 07/13] perf, tools: Add a simple expression parser for  JSON Arnaldo Carvalho de Melo <acme@kernel.org> - 2017-03-21 20:20 +0100
        Re: [PATCH 07/13] perf, tools: Add a simple expression parser for  JSON Arnaldo Carvalho de Melo <acme@kernel.org> - 2017-03-21 20:20 +0100
          Build errors, was Re: [PATCH 07/13] perf, tools: Add a simple  expression parser for JSON Arnaldo Carvalho de Melo <acme@kernel.org> - 2017-03-21 23:10 +0100
      [tip:perf/core] perf tools: Add a simple expression parser for JSON tip-bot for Andi Kleen <tipbot@zytor.com> - 2017-03-24 20:00 +0100
    Re: perf: Improve support for uncore JSON event lists Jiri Olsa <jolsa@redhat.com> - 2017-03-21 15:50 +0100
      Re: perf: Improve support for uncore JSON event lists Arnaldo Carvalho de Melo <acme@kernel.org> - 2017-03-21 21:00 +0100

Page 2 of 2 — ← Prev page 1 [2]

#1608883 — [tip:perf/core] perf list: Move extra details printing to new option

Fromtip-bot for Andi Kleen <tipbot@zytor.com>
Date2017-03-24 20:00 +0100
Subject[tip:perf/core] perf list: Move extra details printing to new option
Message-ID<toCBk-5D8-29@gated-at.bofh.it>
In reply to#1605121
Commit-ID:  bf874fcf9f2fed58510dc83abcee388cee2b427e
Gitweb:     http://git.kernel.org/tip/bf874fcf9f2fed58510dc83abcee388cee2b427e
Author:     Andi Kleen <ak@linux.intel.com>
AuthorDate: Mon, 20 Mar 2017 13:17:11 -0700
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Thu, 23 Mar 2017 11:42:31 -0300

perf list: Move extra details printing to new option

Move the printing of perf expressions and internal events to a new
clearer --details flag, instead of lumping it together with other debug
options in --debug. This makes it clearer to use.

Before

  perf list --debug
  ...
  unc_m_power_critical_throttle_cycles
         [Cycles all ranks are in critical thermal throttle. Unit: uncore_imc]
          uncore_imc_2/event=0x86/  MetricName: power_critical_throttle_cycles % MetricExpr: (unc_m_power_critical_throttle_cycles / unc_m_clockticks) * 100.

after

  perf list --details
  ...
  unc_m_power_critical_throttle_cycles
         [Cycles all ranks are in critical thermal throttle. Unit: uncore_imc]
          uncore_imc_2/event=0x86/  MetricName: power_critical_throttle_cycles % MetricExpr: (unc_m_power_critical_throttle_cycles / unc_m_clockticks) * 100.

Signed-off-by: Andi Kleen <ak@linux.intel.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Link: http://lkml.kernel.org/r/20170320201711.14142-14-andi@firstfloor.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/Documentation/perf-list.txt |  4 ++++
 tools/perf/builtin-list.c              | 14 ++++++++++----
 tools/perf/util/parse-events.c         |  5 +++--
 tools/perf/util/parse-events.h         |  2 +-
 tools/perf/util/pmu.c                  |  4 ++--
 tools/perf/util/pmu.h                  |  2 +-
 6 files changed, 21 insertions(+), 10 deletions(-)

diff --git a/tools/perf/Documentation/perf-list.txt b/tools/perf/Documentation/perf-list.txt
index 41857cc..143d98d 100644
--- a/tools/perf/Documentation/perf-list.txt
+++ b/tools/perf/Documentation/perf-list.txt
@@ -24,6 +24,10 @@ Don't print descriptions.
 --long-desc::
 Print longer event descriptions.
 
+--details::
+Print how named events are resolved internally into perf events, and also
+any extra expressions computed by perf stat.
+
 
 [[EVENT_MODIFIERS]]
 EVENT MODIFIERS
diff --git a/tools/perf/builtin-list.c b/tools/perf/builtin-list.c
index 3b9d98b..be9195e 100644
--- a/tools/perf/builtin-list.c
+++ b/tools/perf/builtin-list.c
@@ -18,6 +18,7 @@
 #include <subcmd/parse-options.h>
 
 static bool desc_flag = true;
+static bool details_flag;
 
 int cmd_list(int argc, const char **argv, const char *prefix __maybe_unused)
 {
@@ -30,6 +31,8 @@ int cmd_list(int argc, const char **argv, const char *prefix __maybe_unused)
 			    "Print extra event descriptions. --no-desc to not print."),
 		OPT_BOOLEAN('v', "long-desc", &long_desc_flag,
 			    "Print longer event descriptions."),
+		OPT_BOOLEAN(0, "details", &details_flag,
+			    "Print information on the perf event names and expressions used internally by events."),
 		OPT_INCR(0, "debug", &verbose,
 			     "Enable debugging output"),
 		OPT_END()
@@ -50,7 +53,8 @@ int cmd_list(int argc, const char **argv, const char *prefix __maybe_unused)
 		printf("\nList of pre-defined events (to be used in -e):\n\n");
 
 	if (argc == 0) {
-		print_events(NULL, raw_dump, !desc_flag, long_desc_flag);
+		print_events(NULL, raw_dump, !desc_flag, long_desc_flag,
+				details_flag);
 		return 0;
 	}
 
@@ -72,7 +76,7 @@ int cmd_list(int argc, const char **argv, const char *prefix __maybe_unused)
 			print_hwcache_events(NULL, raw_dump);
 		else if (strcmp(argv[i], "pmu") == 0)
 			print_pmu_events(NULL, raw_dump, !desc_flag,
-						long_desc_flag);
+						long_desc_flag, details_flag);
 		else if (strcmp(argv[i], "sdt") == 0)
 			print_sdt_events(NULL, NULL, raw_dump);
 		else if ((sep = strchr(argv[i], ':')) != NULL) {
@@ -80,7 +84,8 @@ int cmd_list(int argc, const char **argv, const char *prefix __maybe_unused)
 
 			if (sep == NULL) {
 				print_events(argv[i], raw_dump, !desc_flag,
-							long_desc_flag);
+							long_desc_flag,
+							details_flag);
 				continue;
 			}
 			sep_idx = sep - argv[i];
@@ -103,7 +108,8 @@ int cmd_list(int argc, const char **argv, const char *prefix __maybe_unused)
 					    event_symbols_sw, PERF_COUNT_SW_MAX, raw_dump);
 			print_hwcache_events(s, raw_dump);
 			print_pmu_events(s, raw_dump, !desc_flag,
-						long_desc_flag);
+						long_desc_flag,
+						details_flag);
 			print_tracepoint_events(NULL, s, raw_dump);
 			print_sdt_events(NULL, s, raw_dump);
 			free(s);
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index 119eb0b..6b498ae 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -2325,7 +2325,7 @@ out_enomem:
  * Print the help text for the event symbols:
  */
 void print_events(const char *event_glob, bool name_only, bool quiet_flag,
-			bool long_desc)
+			bool long_desc, bool details_flag)
 {
 	print_symbol_events(event_glob, PERF_TYPE_HARDWARE,
 			    event_symbols_hw, PERF_COUNT_HW_MAX, name_only);
@@ -2335,7 +2335,8 @@ void print_events(const char *event_glob, bool name_only, bool quiet_flag,
 
 	print_hwcache_events(event_glob, name_only);
 
-	print_pmu_events(event_glob, name_only, quiet_flag, long_desc);
+	print_pmu_events(event_glob, name_only, quiet_flag, long_desc,
+			details_flag);
 
 	if (event_glob != NULL)
 		return;
diff --git a/tools/perf/util/parse-events.h b/tools/perf/util/parse-events.h
index f38086b..a235f4d 100644
--- a/tools/perf/util/parse-events.h
+++ b/tools/perf/util/parse-events.h
@@ -184,7 +184,7 @@ void parse_events_evlist_error(struct parse_events_evlist *data,
 			       int idx, const char *str);
 
 void print_events(const char *event_glob, bool name_only, bool quiet,
-		  bool long_desc);
+		  bool long_desc, bool details_flag);
 
 struct event_symbol {
 	const char	*symbol;
diff --git a/tools/perf/util/pmu.c b/tools/perf/util/pmu.c
index bcf752f..362051e 100644
--- a/tools/perf/util/pmu.c
+++ b/tools/perf/util/pmu.c
@@ -1154,7 +1154,7 @@ static void wordwrap(char *s, int start, int max, int corr)
 }
 
 void print_pmu_events(const char *event_glob, bool name_only, bool quiet_flag,
-			bool long_desc)
+			bool long_desc, bool details_flag)
 {
 	struct perf_pmu *pmu;
 	struct perf_pmu_alias *alias;
@@ -1246,7 +1246,7 @@ void print_pmu_events(const char *event_glob, bool name_only, bool quiet_flag,
 			printf("%*s", 8, "[");
 			wordwrap(aliases[j].desc, 8, columns, 0);
 			printf("]\n");
-			if (verbose > 0) {
+			if (details_flag) {
 				printf("%*s%s/%s/ ", 8, "", aliases[j].pmu, aliases[j].str);
 				if (aliases[j].metric_name)
 					printf(" MetricName: %s", aliases[j].metric_name);
diff --git a/tools/perf/util/pmu.h b/tools/perf/util/pmu.h
index 3d4b703..ea7f450 100644
--- a/tools/perf/util/pmu.h
+++ b/tools/perf/util/pmu.h
@@ -80,7 +80,7 @@ int perf_pmu__format_parse(char *dir, struct list_head *head);
 struct perf_pmu *perf_pmu__scan(struct perf_pmu *pmu);
 
 void print_pmu_events(const char *event_glob, bool name_only, bool quiet,
-		      bool long_desc);
+		      bool long_desc, bool details_flag);
 bool pmu_have_event(const char *pname, const char *name);
 
 int perf_pmu__scan_file(struct perf_pmu *pmu, const char *name, const char *fmt,

[toc] | [prev] | [next] | [standalone]

#1605122 — [PATCH 07/13] perf, tools: Add a simple expression parser for JSON

FromAndi Kleen <andi@firstfloor.org>
Date2017-03-20 21:30 +0100
Subject[PATCH 07/13] perf, tools: Add a simple expression parser for JSON
Message-ID<tnc6f-1uQ-23@gated-at.bofh.it>
In reply to#1605110
From: Andi Kleen <ak@linux.intel.com>

Add a simple expression parser good enough to parse JSON relation
expressions. The parser is implemented using bison.

This is just intended as an simple parser for internal usage
in the event lists, not the beginning of a "perf scripting language"

v2: Use expr__ prefix instead of expr_
Support multiple free variables for parser
Signed-off-by: Andi Kleen <ak@linux.intel.com>
---
 tools/perf/tests/Build          |   1 +
 tools/perf/tests/builtin-test.c |   4 +
 tools/perf/tests/expr.c         |  55 +++++++++++++
 tools/perf/tests/tests.h        |   1 +
 tools/perf/util/Build           |   5 ++
 tools/perf/util/expr.h          |  25 ++++++
 tools/perf/util/expr.y          | 173 ++++++++++++++++++++++++++++++++++++++++
 7 files changed, 264 insertions(+)
 create mode 100644 tools/perf/tests/expr.c
 create mode 100644 tools/perf/util/expr.h
 create mode 100644 tools/perf/util/expr.y

diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
index 1cb3d9b540e9..af58ebc243ef 100644
--- a/tools/perf/tests/Build
+++ b/tools/perf/tests/Build
@@ -38,6 +38,7 @@ perf-y += cpumap.o
 perf-y += stat.o
 perf-y += event_update.o
 perf-y += event-times.o
+perf-y += expr.o
 perf-y += backward-ring-buffer.o
 perf-y += sdt.o
 perf-y += is_printable_array.o
diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c
index 83c4669cbc5b..86822969e8a8 100644
--- a/tools/perf/tests/builtin-test.c
+++ b/tools/perf/tests/builtin-test.c
@@ -44,6 +44,10 @@ static struct test generic_tests[] = {
 		.func = test__parse_events,
 	},
 	{
+		.desc = "Simple expression parser",
+		.func = test__expr,
+	},
+	{
 		.desc = "PERF_RECORD_* events & perf_sample fields",
 		.func = test__PERF_RECORD,
 	},
diff --git a/tools/perf/tests/expr.c b/tools/perf/tests/expr.c
new file mode 100644
index 000000000000..554695c06c5b
--- /dev/null
+++ b/tools/perf/tests/expr.c
@@ -0,0 +1,55 @@
+#include "util/debug.h"
+#include "util/expr.h"
+#include "tests.h"
+
+static int test(struct parse_ctx *ctx, const char *e, double val2)
+{
+	double val;
+
+	if (expr__parse(&val, ctx, &e))
+		TEST_ASSERT_VAL("parse test failed", 0);
+	TEST_ASSERT_VAL("unexpected value", val == val2);
+	return 0;
+}
+
+int test__expr(int subtest __maybe_unused)
+{
+	const char *p;
+	const char **other;
+	double val;
+	int ret;
+	struct parse_ctx ctx;
+	int num_other;
+
+	expr__ctx_init(&ctx);
+	expr__add_id(&ctx, "FOO", 1);
+	expr__add_id(&ctx, "BAR", 2);
+
+	ret = test(&ctx, "1+1", 2);
+	ret |= test(&ctx, "FOO+BAR", 3);
+	ret |= test(&ctx, "(BAR/2)%2", 1);
+	ret |= test(&ctx, "1 - -4",  5);
+	ret |= test(&ctx, "(FOO-1)*2 + (BAR/2)%2 - -4",  5);
+
+	if (ret)
+		return ret;
+
+	p = "FOO/0";
+	ret = expr__parse(&val, &ctx, &p);
+	TEST_ASSERT_VAL("division by zero", ret == 1);
+
+	p = "BAR/";
+	ret = expr__parse(&val, &ctx, &p);
+	TEST_ASSERT_VAL("missing operand", ret == 1);
+
+	TEST_ASSERT_VAL("find other",
+			expr__find_other("FOO + BAR + BAZ + BOZO", "FOO", &other, &num_other) == 0);
+	TEST_ASSERT_VAL("find other", num_other == 3);
+	TEST_ASSERT_VAL("find other", !strcmp(other[0], "BAR"));
+	TEST_ASSERT_VAL("find other", !strcmp(other[1], "BAZ"));
+	TEST_ASSERT_VAL("find other", !strcmp(other[2], "BOZO"));
+	TEST_ASSERT_VAL("find other", other[3] == NULL);
+	free((void *)other);
+
+	return 0;
+}
diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h
index 1fa9b9d83aa5..631859629403 100644
--- a/tools/perf/tests/tests.h
+++ b/tools/perf/tests/tests.h
@@ -62,6 +62,7 @@ int test__sample_parsing(int subtest);
 int test__keep_tracking(int subtest);
 int test__parse_no_sample_id_all(int subtest);
 int test__dwarf_unwind(int subtest);
+int test__expr(int subtest);
 int test__hists_filter(int subtest);
 int test__mmap_thread_lookup(int subtest);
 int test__thread_mg_share(int subtest);
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index fb4f42f1bb38..0b98534a9ea1 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -90,6 +90,7 @@ libperf-y += mem-events.o
 libperf-y += vsprintf.o
 libperf-y += drv_configs.o
 libperf-y += time-utils.o
+libperf-y += expr-bison.o
 
 libperf-$(CONFIG_LIBBPF) += bpf-loader.o
 libperf-$(CONFIG_BPF_PROLOGUE) += bpf-prologue.o
@@ -142,6 +143,10 @@ $(OUTPUT)util/parse-events-bison.c: util/parse-events.y
 	$(call rule_mkdir)
 	$(Q)$(call echo-cmd,bison)$(BISON) -v util/parse-events.y -d $(PARSER_DEBUG_BISON) -o $@ -p parse_events_
 
+$(OUTPUT)util/expr-bison.c: util/expr.y
+	$(call rule_mkdir)
+	$(Q)$(call echo-cmd,bison)$(BISON) -v util/expr.y -d $(PARSER_DEBUG_BISON) -o $@ -p expr__
+
 $(OUTPUT)util/pmu-flex.c: util/pmu.l $(OUTPUT)util/pmu-bison.c
 	$(call rule_mkdir)
 	$(Q)$(call echo-cmd,flex)$(FLEX) -o $@ --header-file=$(OUTPUT)util/pmu-flex.h util/pmu.l
diff --git a/tools/perf/util/expr.h b/tools/perf/util/expr.h
new file mode 100644
index 000000000000..9c2760a1a96e
--- /dev/null
+++ b/tools/perf/util/expr.h
@@ -0,0 +1,25 @@
+#ifndef PARSE_CTX_H
+#define PARSE_CTX_H 1
+
+#define EXPR_MAX_OTHER 8
+#define MAX_PARSE_ID EXPR_MAX_OTHER
+
+struct parse_id {
+	const char *name;
+	double val;
+};
+
+struct parse_ctx {
+	int num_ids;
+	struct parse_id ids[MAX_PARSE_ID];
+};
+
+void expr__ctx_init(struct parse_ctx *ctx);
+void expr__add_id(struct parse_ctx *ctx, const char *id, double val);
+#ifndef IN_EXPR_Y
+int expr__parse(double *final_val, struct parse_ctx *ctx, const char **pp);
+#endif
+int expr__find_other(const char *p, const char *one, const char ***other,
+		int *num_other);
+
+#endif
diff --git a/tools/perf/util/expr.y b/tools/perf/util/expr.y
new file mode 100644
index 000000000000..a8daa7a93605
--- /dev/null
+++ b/tools/perf/util/expr.y
@@ -0,0 +1,173 @@
+/* Simple expression parser */
+%{
+#include "util.h"
+#include "util/debug.h"
+#define IN_EXPR_Y 1
+#include "expr.h"
+#include <string.h>
+
+#define MAXIDLEN 256
+%}
+
+%define api.pure full
+%parse-param { double *final_val }
+%parse-param { struct parse_ctx *ctx }
+%parse-param { const char **pp }
+%lex-param { const char **pp }
+
+%union {
+	double num;
+	char id[MAXIDLEN+1];
+}
+
+%token <num> NUMBER
+%token <id> ID
+%left '|'
+%left '^'
+%left '&'
+%left '-' '+'
+%left '*' '/' '%'
+%left NEG NOT
+%type <num> expr
+
+%{
+static int expr__lex(YYSTYPE *res, const char **pp);
+
+static void expr__error(double *final_val __maybe_unused,
+		       struct parse_ctx *ctx __maybe_unused,
+		       const char **pp __maybe_unused,
+		       const char *s)
+{
+	pr_debug("%s\n", s);
+}
+
+static int lookup_id(struct parse_ctx *ctx, char *id, double *val)
+{
+	int i;
+
+	for (i = 0; i < ctx->num_ids; i++) {
+		if (!strcasecmp(ctx->ids[i].name, id)) {
+			*val = ctx->ids[i].val;
+			return 0;
+		}
+	}
+	return -1;
+}
+
+%}
+%%
+
+all_expr: expr			{ *final_val = $1; }
+	;
+
+expr:	  NUMBER
+	| ID			{ if (lookup_id(ctx, $1, &$$) < 0) {
+					pr_debug("%s not found", $1);
+					YYABORT;
+				  }
+				}
+	| expr '+' expr		{ $$ = $1 + $3; }
+	| expr '-' expr		{ $$ = $1 - $3; }
+	| expr '*' expr		{ $$ = $1 * $3; }
+	| expr '/' expr		{ if ($3 == 0) YYABORT; $$ = $1 / $3; }
+	| expr '%' expr		{ if ((long)$3 == 0) YYABORT; $$ = (long)$1 % (long)$3; }
+	| '-' expr %prec NEG	{ $$ = -$2; }
+	| '(' expr ')'		{ $$ = $2; }
+	;
+
+%%
+
+static int expr__symbol(YYSTYPE *res, const char *p, const char **pp)
+{
+	char *dst = res->id;
+	const char *s = p;
+
+	while (isalnum(*p) || *p == '_' || *p == '.') {
+		if (p - s >= MAXIDLEN)
+			return -1;
+		*dst++ = *p++;
+	}
+	*dst = 0;
+	*pp = p;
+	return ID;
+}
+
+static int expr__lex(YYSTYPE *res, const char **pp)
+{
+	int tok;
+	const char *s;
+	const char *p = *pp;
+
+	while (isspace(*p))
+		p++;
+	s = p;
+	switch (*p++) {
+	case 'a' ... 'z':
+	case 'A' ... 'Z':
+		return expr__symbol(res, p - 1, pp);
+	case '0' ... '9': case '.':
+		res->num = strtod(s, (char **)&p);
+		tok = NUMBER;
+		break;
+	default:
+		tok = *s;
+		break;
+	}
+	*pp = p;
+	return tok;
+}
+
+/* Caller must make sure id is allocated */
+void expr__add_id(struct parse_ctx *ctx, const char *name, double val)
+{
+	int idx;
+	assert(ctx->num_ids < MAX_PARSE_ID);
+	idx = ctx->num_ids++;
+	ctx->ids[idx].name = name;
+	ctx->ids[idx].val = val;
+}
+
+void expr__ctx_init(struct parse_ctx *ctx)
+{
+	ctx->num_ids = 0;
+}
+
+int expr__find_other(const char *p, const char *one, const char ***other,
+		     int *num_otherp)
+{
+	const char *orig = p;
+	int err = -1;
+	int num_other;
+
+	*other = malloc((EXPR_MAX_OTHER + 1) * sizeof(char *));
+	if (!*other)
+		return -1;
+
+	num_other = 0;
+	for (;;) {
+		YYSTYPE val;
+		int tok = expr__lex(&val, &p);
+		if (tok == 0) {
+			err = 0;
+			break;
+		}
+		if (tok == ID && strcasecmp(one, val.id)) {
+			if (num_other >= EXPR_MAX_OTHER - 1) {
+				pr_debug("Too many extra events in %s\n", orig);
+				break;
+			}
+			(*other)[num_other] = strdup(val.id);
+			if (!(*other)[num_other])
+				return -1;
+			num_other++;
+		}
+	}
+	(*other)[num_other] = NULL;
+	*num_otherp = num_other;
+	if (err) {
+		*num_otherp = 0;
+		free(*other);
+		*other = NULL;
+	}
+	return err;
+}
-- 
2.9.3

[toc] | [prev] | [next] | [standalone]

#1605929 — Re: [PATCH 07/13] perf, tools: Add a simple expression parser for JSON

FromArnaldo Carvalho de Melo <acme@kernel.org>
Date2017-03-21 20:20 +0100
SubjectRe: [PATCH 07/13] perf, tools: Add a simple expression parser for JSON
Message-ID<tnxu2-7Db-25@gated-at.bofh.it>
In reply to#1605122
Em Mon, Mar 20, 2017 at 01:17:05PM -0700, Andi Kleen escreveu:
> From: Andi Kleen <ak@linux.intel.com>
> 
> Add a simple expression parser good enough to parse JSON relation
> expressions. The parser is implemented using bison.

Renaming 'foo_expr' to 'foo_json_expr', as 'expr' is _way_ too
generic...

- Arnaldo
 
> This is just intended as an simple parser for internal usage
> in the event lists, not the beginning of a "perf scripting language"
> 
> v2: Use expr__ prefix instead of expr_
> Support multiple free variables for parser
> Signed-off-by: Andi Kleen <ak@linux.intel.com>
> ---
>  tools/perf/tests/Build          |   1 +
>  tools/perf/tests/builtin-test.c |   4 +
>  tools/perf/tests/expr.c         |  55 +++++++++++++
>  tools/perf/tests/tests.h        |   1 +
>  tools/perf/util/Build           |   5 ++
>  tools/perf/util/expr.h          |  25 ++++++
>  tools/perf/util/expr.y          | 173 ++++++++++++++++++++++++++++++++++++++++
>  7 files changed, 264 insertions(+)
>  create mode 100644 tools/perf/tests/expr.c
>  create mode 100644 tools/perf/util/expr.h
>  create mode 100644 tools/perf/util/expr.y
> 
> diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
> index 1cb3d9b540e9..af58ebc243ef 100644
> --- a/tools/perf/tests/Build
> +++ b/tools/perf/tests/Build
> @@ -38,6 +38,7 @@ perf-y += cpumap.o
>  perf-y += stat.o
>  perf-y += event_update.o
>  perf-y += event-times.o
> +perf-y += expr.o
>  perf-y += backward-ring-buffer.o
>  perf-y += sdt.o
>  perf-y += is_printable_array.o
> diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c
> index 83c4669cbc5b..86822969e8a8 100644
> --- a/tools/perf/tests/builtin-test.c
> +++ b/tools/perf/tests/builtin-test.c
> @@ -44,6 +44,10 @@ static struct test generic_tests[] = {
>  		.func = test__parse_events,
>  	},
>  	{
> +		.desc = "Simple expression parser",
> +		.func = test__expr,
> +	},
> +	{
>  		.desc = "PERF_RECORD_* events & perf_sample fields",
>  		.func = test__PERF_RECORD,
>  	},
> diff --git a/tools/perf/tests/expr.c b/tools/perf/tests/expr.c
> new file mode 100644
> index 000000000000..554695c06c5b
> --- /dev/null
> +++ b/tools/perf/tests/expr.c
> @@ -0,0 +1,55 @@
> +#include "util/debug.h"
> +#include "util/expr.h"
> +#include "tests.h"
> +
> +static int test(struct parse_ctx *ctx, const char *e, double val2)
> +{
> +	double val;
> +
> +	if (expr__parse(&val, ctx, &e))
> +		TEST_ASSERT_VAL("parse test failed", 0);
> +	TEST_ASSERT_VAL("unexpected value", val == val2);
> +	return 0;
> +}
> +
> +int test__expr(int subtest __maybe_unused)
> +{
> +	const char *p;
> +	const char **other;
> +	double val;
> +	int ret;
> +	struct parse_ctx ctx;
> +	int num_other;
> +
> +	expr__ctx_init(&ctx);
> +	expr__add_id(&ctx, "FOO", 1);
> +	expr__add_id(&ctx, "BAR", 2);
> +
> +	ret = test(&ctx, "1+1", 2);
> +	ret |= test(&ctx, "FOO+BAR", 3);
> +	ret |= test(&ctx, "(BAR/2)%2", 1);
> +	ret |= test(&ctx, "1 - -4",  5);
> +	ret |= test(&ctx, "(FOO-1)*2 + (BAR/2)%2 - -4",  5);
> +
> +	if (ret)
> +		return ret;
> +
> +	p = "FOO/0";
> +	ret = expr__parse(&val, &ctx, &p);
> +	TEST_ASSERT_VAL("division by zero", ret == 1);
> +
> +	p = "BAR/";
> +	ret = expr__parse(&val, &ctx, &p);
> +	TEST_ASSERT_VAL("missing operand", ret == 1);
> +
> +	TEST_ASSERT_VAL("find other",
> +			expr__find_other("FOO + BAR + BAZ + BOZO", "FOO", &other, &num_other) == 0);
> +	TEST_ASSERT_VAL("find other", num_other == 3);
> +	TEST_ASSERT_VAL("find other", !strcmp(other[0], "BAR"));
> +	TEST_ASSERT_VAL("find other", !strcmp(other[1], "BAZ"));
> +	TEST_ASSERT_VAL("find other", !strcmp(other[2], "BOZO"));
> +	TEST_ASSERT_VAL("find other", other[3] == NULL);
> +	free((void *)other);
> +
> +	return 0;
> +}
> diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h
> index 1fa9b9d83aa5..631859629403 100644
> --- a/tools/perf/tests/tests.h
> +++ b/tools/perf/tests/tests.h
> @@ -62,6 +62,7 @@ int test__sample_parsing(int subtest);
>  int test__keep_tracking(int subtest);
>  int test__parse_no_sample_id_all(int subtest);
>  int test__dwarf_unwind(int subtest);
> +int test__expr(int subtest);
>  int test__hists_filter(int subtest);
>  int test__mmap_thread_lookup(int subtest);
>  int test__thread_mg_share(int subtest);
> diff --git a/tools/perf/util/Build b/tools/perf/util/Build
> index fb4f42f1bb38..0b98534a9ea1 100644
> --- a/tools/perf/util/Build
> +++ b/tools/perf/util/Build
> @@ -90,6 +90,7 @@ libperf-y += mem-events.o
>  libperf-y += vsprintf.o
>  libperf-y += drv_configs.o
>  libperf-y += time-utils.o
> +libperf-y += expr-bison.o
>  
>  libperf-$(CONFIG_LIBBPF) += bpf-loader.o
>  libperf-$(CONFIG_BPF_PROLOGUE) += bpf-prologue.o
> @@ -142,6 +143,10 @@ $(OUTPUT)util/parse-events-bison.c: util/parse-events.y
>  	$(call rule_mkdir)
>  	$(Q)$(call echo-cmd,bison)$(BISON) -v util/parse-events.y -d $(PARSER_DEBUG_BISON) -o $@ -p parse_events_
>  
> +$(OUTPUT)util/expr-bison.c: util/expr.y
> +	$(call rule_mkdir)
> +	$(Q)$(call echo-cmd,bison)$(BISON) -v util/expr.y -d $(PARSER_DEBUG_BISON) -o $@ -p expr__
> +
>  $(OUTPUT)util/pmu-flex.c: util/pmu.l $(OUTPUT)util/pmu-bison.c
>  	$(call rule_mkdir)
>  	$(Q)$(call echo-cmd,flex)$(FLEX) -o $@ --header-file=$(OUTPUT)util/pmu-flex.h util/pmu.l
> diff --git a/tools/perf/util/expr.h b/tools/perf/util/expr.h
> new file mode 100644
> index 000000000000..9c2760a1a96e
> --- /dev/null
> +++ b/tools/perf/util/expr.h
> @@ -0,0 +1,25 @@
> +#ifndef PARSE_CTX_H
> +#define PARSE_CTX_H 1
> +
> +#define EXPR_MAX_OTHER 8
> +#define MAX_PARSE_ID EXPR_MAX_OTHER
> +
> +struct parse_id {
> +	const char *name;
> +	double val;
> +};
> +
> +struct parse_ctx {
> +	int num_ids;
> +	struct parse_id ids[MAX_PARSE_ID];
> +};
> +
> +void expr__ctx_init(struct parse_ctx *ctx);
> +void expr__add_id(struct parse_ctx *ctx, const char *id, double val);
> +#ifndef IN_EXPR_Y
> +int expr__parse(double *final_val, struct parse_ctx *ctx, const char **pp);
> +#endif
> +int expr__find_other(const char *p, const char *one, const char ***other,
> +		int *num_other);
> +
> +#endif
> diff --git a/tools/perf/util/expr.y b/tools/perf/util/expr.y
> new file mode 100644
> index 000000000000..a8daa7a93605
> --- /dev/null
> +++ b/tools/perf/util/expr.y
> @@ -0,0 +1,173 @@
> +/* Simple expression parser */
> +%{
> +#include "util.h"
> +#include "util/debug.h"
> +#define IN_EXPR_Y 1
> +#include "expr.h"
> +#include <string.h>
> +
> +#define MAXIDLEN 256
> +%}
> +
> +%define api.pure full
> +%parse-param { double *final_val }
> +%parse-param { struct parse_ctx *ctx }
> +%parse-param { const char **pp }
> +%lex-param { const char **pp }
> +
> +%union {
> +	double num;
> +	char id[MAXIDLEN+1];
> +}
> +
> +%token <num> NUMBER
> +%token <id> ID
> +%left '|'
> +%left '^'
> +%left '&'
> +%left '-' '+'
> +%left '*' '/' '%'
> +%left NEG NOT
> +%type <num> expr
> +
> +%{
> +static int expr__lex(YYSTYPE *res, const char **pp);
> +
> +static void expr__error(double *final_val __maybe_unused,
> +		       struct parse_ctx *ctx __maybe_unused,
> +		       const char **pp __maybe_unused,
> +		       const char *s)
> +{
> +	pr_debug("%s\n", s);
> +}
> +
> +static int lookup_id(struct parse_ctx *ctx, char *id, double *val)
> +{
> +	int i;
> +
> +	for (i = 0; i < ctx->num_ids; i++) {
> +		if (!strcasecmp(ctx->ids[i].name, id)) {
> +			*val = ctx->ids[i].val;
> +			return 0;
> +		}
> +	}
> +	return -1;
> +}
> +
> +%}
> +%%
> +
> +all_expr: expr			{ *final_val = $1; }
> +	;
> +
> +expr:	  NUMBER
> +	| ID			{ if (lookup_id(ctx, $1, &$$) < 0) {
> +					pr_debug("%s not found", $1);
> +					YYABORT;
> +				  }
> +				}
> +	| expr '+' expr		{ $$ = $1 + $3; }
> +	| expr '-' expr		{ $$ = $1 - $3; }
> +	| expr '*' expr		{ $$ = $1 * $3; }
> +	| expr '/' expr		{ if ($3 == 0) YYABORT; $$ = $1 / $3; }
> +	| expr '%' expr		{ if ((long)$3 == 0) YYABORT; $$ = (long)$1 % (long)$3; }
> +	| '-' expr %prec NEG	{ $$ = -$2; }
> +	| '(' expr ')'		{ $$ = $2; }
> +	;
> +
> +%%
> +
> +static int expr__symbol(YYSTYPE *res, const char *p, const char **pp)
> +{
> +	char *dst = res->id;
> +	const char *s = p;
> +
> +	while (isalnum(*p) || *p == '_' || *p == '.') {
> +		if (p - s >= MAXIDLEN)
> +			return -1;
> +		*dst++ = *p++;
> +	}
> +	*dst = 0;
> +	*pp = p;
> +	return ID;
> +}
> +
> +static int expr__lex(YYSTYPE *res, const char **pp)
> +{
> +	int tok;
> +	const char *s;
> +	const char *p = *pp;
> +
> +	while (isspace(*p))
> +		p++;
> +	s = p;
> +	switch (*p++) {
> +	case 'a' ... 'z':
> +	case 'A' ... 'Z':
> +		return expr__symbol(res, p - 1, pp);
> +	case '0' ... '9': case '.':
> +		res->num = strtod(s, (char **)&p);
> +		tok = NUMBER;
> +		break;
> +	default:
> +		tok = *s;
> +		break;
> +	}
> +	*pp = p;
> +	return tok;
> +}
> +
> +/* Caller must make sure id is allocated */
> +void expr__add_id(struct parse_ctx *ctx, const char *name, double val)
> +{
> +	int idx;
> +	assert(ctx->num_ids < MAX_PARSE_ID);
> +	idx = ctx->num_ids++;
> +	ctx->ids[idx].name = name;
> +	ctx->ids[idx].val = val;
> +}
> +
> +void expr__ctx_init(struct parse_ctx *ctx)
> +{
> +	ctx->num_ids = 0;
> +}
> +
> +int expr__find_other(const char *p, const char *one, const char ***other,
> +		     int *num_otherp)
> +{
> +	const char *orig = p;
> +	int err = -1;
> +	int num_other;
> +
> +	*other = malloc((EXPR_MAX_OTHER + 1) * sizeof(char *));
> +	if (!*other)
> +		return -1;
> +
> +	num_other = 0;
> +	for (;;) {
> +		YYSTYPE val;
> +		int tok = expr__lex(&val, &p);
> +		if (tok == 0) {
> +			err = 0;
> +			break;
> +		}
> +		if (tok == ID && strcasecmp(one, val.id)) {
> +			if (num_other >= EXPR_MAX_OTHER - 1) {
> +				pr_debug("Too many extra events in %s\n", orig);
> +				break;
> +			}
> +			(*other)[num_other] = strdup(val.id);
> +			if (!(*other)[num_other])
> +				return -1;
> +			num_other++;
> +		}
> +	}
> +	(*other)[num_other] = NULL;
> +	*num_otherp = num_other;
> +	if (err) {
> +		*num_otherp = 0;
> +		free(*other);
> +		*other = NULL;
> +	}
> +	return err;
> +}
> -- 
> 2.9.3

[toc] | [prev] | [next] | [standalone]

#1605930 — Re: [PATCH 07/13] perf, tools: Add a simple expression parser for JSON

FromArnaldo Carvalho de Melo <acme@kernel.org>
Date2017-03-21 20:20 +0100
SubjectRe: [PATCH 07/13] perf, tools: Add a simple expression parser for JSON
Message-ID<tnxu2-7Db-33@gated-at.bofh.it>
In reply to#1605929
Em Tue, Mar 21, 2017 at 04:14:23PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Mon, Mar 20, 2017 at 01:17:05PM -0700, Andi Kleen escreveu:
> > From: Andi Kleen <ak@linux.intel.com>
> > 
> > Add a simple expression parser good enough to parse JSON relation
> > expressions. The parser is implemented using bison.
> 
> Renaming 'foo_expr' to 'foo_json_expr', as 'expr' is _way_ too
> generic...

I retract that, it is used with JSON, but isn't strictly tied to it,
keeping as is, nevermind.

- Arnaldo
 
> - Arnaldo
>  
> > This is just intended as an simple parser for internal usage
> > in the event lists, not the beginning of a "perf scripting language"
> > 
> > v2: Use expr__ prefix instead of expr_
> > Support multiple free variables for parser
> > Signed-off-by: Andi Kleen <ak@linux.intel.com>
> > ---
> >  tools/perf/tests/Build          |   1 +
> >  tools/perf/tests/builtin-test.c |   4 +
> >  tools/perf/tests/expr.c         |  55 +++++++++++++
> >  tools/perf/tests/tests.h        |   1 +
> >  tools/perf/util/Build           |   5 ++
> >  tools/perf/util/expr.h          |  25 ++++++
> >  tools/perf/util/expr.y          | 173 ++++++++++++++++++++++++++++++++++++++++
> >  7 files changed, 264 insertions(+)
> >  create mode 100644 tools/perf/tests/expr.c
> >  create mode 100644 tools/perf/util/expr.h
> >  create mode 100644 tools/perf/util/expr.y
> > 
> > diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
> > index 1cb3d9b540e9..af58ebc243ef 100644
> > --- a/tools/perf/tests/Build
> > +++ b/tools/perf/tests/Build
> > @@ -38,6 +38,7 @@ perf-y += cpumap.o
> >  perf-y += stat.o
> >  perf-y += event_update.o
> >  perf-y += event-times.o
> > +perf-y += expr.o
> >  perf-y += backward-ring-buffer.o
> >  perf-y += sdt.o
> >  perf-y += is_printable_array.o
> > diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c
> > index 83c4669cbc5b..86822969e8a8 100644
> > --- a/tools/perf/tests/builtin-test.c
> > +++ b/tools/perf/tests/builtin-test.c
> > @@ -44,6 +44,10 @@ static struct test generic_tests[] = {
> >  		.func = test__parse_events,
> >  	},
> >  	{
> > +		.desc = "Simple expression parser",
> > +		.func = test__expr,
> > +	},
> > +	{
> >  		.desc = "PERF_RECORD_* events & perf_sample fields",
> >  		.func = test__PERF_RECORD,
> >  	},
> > diff --git a/tools/perf/tests/expr.c b/tools/perf/tests/expr.c
> > new file mode 100644
> > index 000000000000..554695c06c5b
> > --- /dev/null
> > +++ b/tools/perf/tests/expr.c
> > @@ -0,0 +1,55 @@
> > +#include "util/debug.h"
> > +#include "util/expr.h"
> > +#include "tests.h"
> > +
> > +static int test(struct parse_ctx *ctx, const char *e, double val2)
> > +{
> > +	double val;
> > +
> > +	if (expr__parse(&val, ctx, &e))
> > +		TEST_ASSERT_VAL("parse test failed", 0);
> > +	TEST_ASSERT_VAL("unexpected value", val == val2);
> > +	return 0;
> > +}
> > +
> > +int test__expr(int subtest __maybe_unused)
> > +{
> > +	const char *p;
> > +	const char **other;
> > +	double val;
> > +	int ret;
> > +	struct parse_ctx ctx;
> > +	int num_other;
> > +
> > +	expr__ctx_init(&ctx);
> > +	expr__add_id(&ctx, "FOO", 1);
> > +	expr__add_id(&ctx, "BAR", 2);
> > +
> > +	ret = test(&ctx, "1+1", 2);
> > +	ret |= test(&ctx, "FOO+BAR", 3);
> > +	ret |= test(&ctx, "(BAR/2)%2", 1);
> > +	ret |= test(&ctx, "1 - -4",  5);
> > +	ret |= test(&ctx, "(FOO-1)*2 + (BAR/2)%2 - -4",  5);
> > +
> > +	if (ret)
> > +		return ret;
> > +
> > +	p = "FOO/0";
> > +	ret = expr__parse(&val, &ctx, &p);
> > +	TEST_ASSERT_VAL("division by zero", ret == 1);
> > +
> > +	p = "BAR/";
> > +	ret = expr__parse(&val, &ctx, &p);
> > +	TEST_ASSERT_VAL("missing operand", ret == 1);
> > +
> > +	TEST_ASSERT_VAL("find other",
> > +			expr__find_other("FOO + BAR + BAZ + BOZO", "FOO", &other, &num_other) == 0);
> > +	TEST_ASSERT_VAL("find other", num_other == 3);
> > +	TEST_ASSERT_VAL("find other", !strcmp(other[0], "BAR"));
> > +	TEST_ASSERT_VAL("find other", !strcmp(other[1], "BAZ"));
> > +	TEST_ASSERT_VAL("find other", !strcmp(other[2], "BOZO"));
> > +	TEST_ASSERT_VAL("find other", other[3] == NULL);
> > +	free((void *)other);
> > +
> > +	return 0;
> > +}
> > diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h
> > index 1fa9b9d83aa5..631859629403 100644
> > --- a/tools/perf/tests/tests.h
> > +++ b/tools/perf/tests/tests.h
> > @@ -62,6 +62,7 @@ int test__sample_parsing(int subtest);
> >  int test__keep_tracking(int subtest);
> >  int test__parse_no_sample_id_all(int subtest);
> >  int test__dwarf_unwind(int subtest);
> > +int test__expr(int subtest);
> >  int test__hists_filter(int subtest);
> >  int test__mmap_thread_lookup(int subtest);
> >  int test__thread_mg_share(int subtest);
> > diff --git a/tools/perf/util/Build b/tools/perf/util/Build
> > index fb4f42f1bb38..0b98534a9ea1 100644
> > --- a/tools/perf/util/Build
> > +++ b/tools/perf/util/Build
> > @@ -90,6 +90,7 @@ libperf-y += mem-events.o
> >  libperf-y += vsprintf.o
> >  libperf-y += drv_configs.o
> >  libperf-y += time-utils.o
> > +libperf-y += expr-bison.o
> >  
> >  libperf-$(CONFIG_LIBBPF) += bpf-loader.o
> >  libperf-$(CONFIG_BPF_PROLOGUE) += bpf-prologue.o
> > @@ -142,6 +143,10 @@ $(OUTPUT)util/parse-events-bison.c: util/parse-events.y
> >  	$(call rule_mkdir)
> >  	$(Q)$(call echo-cmd,bison)$(BISON) -v util/parse-events.y -d $(PARSER_DEBUG_BISON) -o $@ -p parse_events_
> >  
> > +$(OUTPUT)util/expr-bison.c: util/expr.y
> > +	$(call rule_mkdir)
> > +	$(Q)$(call echo-cmd,bison)$(BISON) -v util/expr.y -d $(PARSER_DEBUG_BISON) -o $@ -p expr__
> > +
> >  $(OUTPUT)util/pmu-flex.c: util/pmu.l $(OUTPUT)util/pmu-bison.c
> >  	$(call rule_mkdir)
> >  	$(Q)$(call echo-cmd,flex)$(FLEX) -o $@ --header-file=$(OUTPUT)util/pmu-flex.h util/pmu.l
> > diff --git a/tools/perf/util/expr.h b/tools/perf/util/expr.h
> > new file mode 100644
> > index 000000000000..9c2760a1a96e
> > --- /dev/null
> > +++ b/tools/perf/util/expr.h
> > @@ -0,0 +1,25 @@
> > +#ifndef PARSE_CTX_H
> > +#define PARSE_CTX_H 1
> > +
> > +#define EXPR_MAX_OTHER 8
> > +#define MAX_PARSE_ID EXPR_MAX_OTHER
> > +
> > +struct parse_id {
> > +	const char *name;
> > +	double val;
> > +};
> > +
> > +struct parse_ctx {
> > +	int num_ids;
> > +	struct parse_id ids[MAX_PARSE_ID];
> > +};
> > +
> > +void expr__ctx_init(struct parse_ctx *ctx);
> > +void expr__add_id(struct parse_ctx *ctx, const char *id, double val);
> > +#ifndef IN_EXPR_Y
> > +int expr__parse(double *final_val, struct parse_ctx *ctx, const char **pp);
> > +#endif
> > +int expr__find_other(const char *p, const char *one, const char ***other,
> > +		int *num_other);
> > +
> > +#endif
> > diff --git a/tools/perf/util/expr.y b/tools/perf/util/expr.y
> > new file mode 100644
> > index 000000000000..a8daa7a93605
> > --- /dev/null
> > +++ b/tools/perf/util/expr.y
> > @@ -0,0 +1,173 @@
> > +/* Simple expression parser */
> > +%{
> > +#include "util.h"
> > +#include "util/debug.h"
> > +#define IN_EXPR_Y 1
> > +#include "expr.h"
> > +#include <string.h>
> > +
> > +#define MAXIDLEN 256
> > +%}
> > +
> > +%define api.pure full
> > +%parse-param { double *final_val }
> > +%parse-param { struct parse_ctx *ctx }
> > +%parse-param { const char **pp }
> > +%lex-param { const char **pp }
> > +
> > +%union {
> > +	double num;
> > +	char id[MAXIDLEN+1];
> > +}
> > +
> > +%token <num> NUMBER
> > +%token <id> ID
> > +%left '|'
> > +%left '^'
> > +%left '&'
> > +%left '-' '+'
> > +%left '*' '/' '%'
> > +%left NEG NOT
> > +%type <num> expr
> > +
> > +%{
> > +static int expr__lex(YYSTYPE *res, const char **pp);
> > +
> > +static void expr__error(double *final_val __maybe_unused,
> > +		       struct parse_ctx *ctx __maybe_unused,
> > +		       const char **pp __maybe_unused,
> > +		       const char *s)
> > +{
> > +	pr_debug("%s\n", s);
> > +}
> > +
> > +static int lookup_id(struct parse_ctx *ctx, char *id, double *val)
> > +{
> > +	int i;
> > +
> > +	for (i = 0; i < ctx->num_ids; i++) {
> > +		if (!strcasecmp(ctx->ids[i].name, id)) {
> > +			*val = ctx->ids[i].val;
> > +			return 0;
> > +		}
> > +	}
> > +	return -1;
> > +}
> > +
> > +%}
> > +%%
> > +
> > +all_expr: expr			{ *final_val = $1; }
> > +	;
> > +
> > +expr:	  NUMBER
> > +	| ID			{ if (lookup_id(ctx, $1, &$$) < 0) {
> > +					pr_debug("%s not found", $1);
> > +					YYABORT;
> > +				  }
> > +				}
> > +	| expr '+' expr		{ $$ = $1 + $3; }
> > +	| expr '-' expr		{ $$ = $1 - $3; }
> > +	| expr '*' expr		{ $$ = $1 * $3; }
> > +	| expr '/' expr		{ if ($3 == 0) YYABORT; $$ = $1 / $3; }
> > +	| expr '%' expr		{ if ((long)$3 == 0) YYABORT; $$ = (long)$1 % (long)$3; }
> > +	| '-' expr %prec NEG	{ $$ = -$2; }
> > +	| '(' expr ')'		{ $$ = $2; }
> > +	;
> > +
> > +%%
> > +
> > +static int expr__symbol(YYSTYPE *res, const char *p, const char **pp)
> > +{
> > +	char *dst = res->id;
> > +	const char *s = p;
> > +
> > +	while (isalnum(*p) || *p == '_' || *p == '.') {
> > +		if (p - s >= MAXIDLEN)
> > +			return -1;
> > +		*dst++ = *p++;
> > +	}
> > +	*dst = 0;
> > +	*pp = p;
> > +	return ID;
> > +}
> > +
> > +static int expr__lex(YYSTYPE *res, const char **pp)
> > +{
> > +	int tok;
> > +	const char *s;
> > +	const char *p = *pp;
> > +
> > +	while (isspace(*p))
> > +		p++;
> > +	s = p;
> > +	switch (*p++) {
> > +	case 'a' ... 'z':
> > +	case 'A' ... 'Z':
> > +		return expr__symbol(res, p - 1, pp);
> > +	case '0' ... '9': case '.':
> > +		res->num = strtod(s, (char **)&p);
> > +		tok = NUMBER;
> > +		break;
> > +	default:
> > +		tok = *s;
> > +		break;
> > +	}
> > +	*pp = p;
> > +	return tok;
> > +}
> > +
> > +/* Caller must make sure id is allocated */
> > +void expr__add_id(struct parse_ctx *ctx, const char *name, double val)
> > +{
> > +	int idx;
> > +	assert(ctx->num_ids < MAX_PARSE_ID);
> > +	idx = ctx->num_ids++;
> > +	ctx->ids[idx].name = name;
> > +	ctx->ids[idx].val = val;
> > +}
> > +
> > +void expr__ctx_init(struct parse_ctx *ctx)
> > +{
> > +	ctx->num_ids = 0;
> > +}
> > +
> > +int expr__find_other(const char *p, const char *one, const char ***other,
> > +		     int *num_otherp)
> > +{
> > +	const char *orig = p;
> > +	int err = -1;
> > +	int num_other;
> > +
> > +	*other = malloc((EXPR_MAX_OTHER + 1) * sizeof(char *));
> > +	if (!*other)
> > +		return -1;
> > +
> > +	num_other = 0;
> > +	for (;;) {
> > +		YYSTYPE val;
> > +		int tok = expr__lex(&val, &p);
> > +		if (tok == 0) {
> > +			err = 0;
> > +			break;
> > +		}
> > +		if (tok == ID && strcasecmp(one, val.id)) {
> > +			if (num_other >= EXPR_MAX_OTHER - 1) {
> > +				pr_debug("Too many extra events in %s\n", orig);
> > +				break;
> > +			}
> > +			(*other)[num_other] = strdup(val.id);
> > +			if (!(*other)[num_other])
> > +				return -1;
> > +			num_other++;
> > +		}
> > +	}
> > +	(*other)[num_other] = NULL;
> > +	*num_otherp = num_other;
> > +	if (err) {
> > +		*num_otherp = 0;
> > +		free(*other);
> > +		*other = NULL;
> > +	}
> > +	return err;
> > +}
> > -- 
> > 2.9.3

[toc] | [prev] | [next] | [standalone]

#1606020 — Build errors, was Re: [PATCH 07/13] perf, tools: Add a simple expression parser for JSON

FromArnaldo Carvalho de Melo <acme@kernel.org>
Date2017-03-21 23:10 +0100
SubjectBuild errors, was Re: [PATCH 07/13] perf, tools: Add a simple expression parser for JSON
Message-ID<tnA8y-18H-9@gated-at.bofh.it>
In reply to#1605930
Em Tue, Mar 21, 2017 at 04:15:49PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Tue, Mar 21, 2017 at 04:14:23PM -0300, Arnaldo Carvalho de Melo escreveu:
> > Em Mon, Mar 20, 2017 at 01:17:05PM -0700, Andi Kleen escreveu:
> > > From: Andi Kleen <ak@linux.intel.com>
> > > 
> > > Add a simple expression parser good enough to parse JSON relation
> > > expressions. The parser is implemented using bison.
> > 
> > Renaming 'foo_expr' to 'foo_json_expr', as 'expr' is _way_ too
> > generic...
> 
> I retract that, it is used with JSON, but isn't strictly tied to it,
> keeping as is, nevermind.

In other news, several environments (e.g ubuntu:16.04-x-s390, debian
experimental cross compilers, at least for mips and mips64,
debian:experimental-x-arm64 worked tho) failed with:

tests/expr.c: In function 'test__expr':
tests/expr.c:52:2: error: implicit declaration of function 'free' [-Werror=implicit-function-declaration]
  free((void *)other);
  ^~~~
 
And some (e.g. CentOS5, CentOS6) with:

  BISON    /tmp/build/perf/util/expr-bison.c
util/expr.y:12.9-16: syntax error, unexpected identifier, expecting string
make[4]: *** [/tmp/build/perf/util/expr-bison.c] Error 1

Others (e.g. debian:7, ubuntu:12.04.5) with:

  BISON    /tmp/build/perf/util/expr-bison.c
/usr/share/bison/yacc.c:94: error: invalid pure value: pure
/usr/share/bison/yacc.c:94: the top level
util/expr.y:12.9-16: invalid value for %define Boolean variable `api.pure'
make[4]: *** [/tmp/build/perf/util/expr-bison.c] Error 1

Fixing those, then running the tests again...

- Arnaldo

> - Arnaldo
>  
> > - Arnaldo
> >  
> > > This is just intended as an simple parser for internal usage
> > > in the event lists, not the beginning of a "perf scripting language"
> > > 
> > > v2: Use expr__ prefix instead of expr_
> > > Support multiple free variables for parser
> > > Signed-off-by: Andi Kleen <ak@linux.intel.com>
> > > ---
> > >  tools/perf/tests/Build          |   1 +
> > >  tools/perf/tests/builtin-test.c |   4 +
> > >  tools/perf/tests/expr.c         |  55 +++++++++++++
> > >  tools/perf/tests/tests.h        |   1 +
> > >  tools/perf/util/Build           |   5 ++
> > >  tools/perf/util/expr.h          |  25 ++++++
> > >  tools/perf/util/expr.y          | 173 ++++++++++++++++++++++++++++++++++++++++
> > >  7 files changed, 264 insertions(+)
> > >  create mode 100644 tools/perf/tests/expr.c
> > >  create mode 100644 tools/perf/util/expr.h
> > >  create mode 100644 tools/perf/util/expr.y
> > > 
> > > diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
> > > index 1cb3d9b540e9..af58ebc243ef 100644
> > > --- a/tools/perf/tests/Build
> > > +++ b/tools/perf/tests/Build
> > > @@ -38,6 +38,7 @@ perf-y += cpumap.o
> > >  perf-y += stat.o
> > >  perf-y += event_update.o
> > >  perf-y += event-times.o
> > > +perf-y += expr.o
> > >  perf-y += backward-ring-buffer.o
> > >  perf-y += sdt.o
> > >  perf-y += is_printable_array.o
> > > diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c
> > > index 83c4669cbc5b..86822969e8a8 100644
> > > --- a/tools/perf/tests/builtin-test.c
> > > +++ b/tools/perf/tests/builtin-test.c
> > > @@ -44,6 +44,10 @@ static struct test generic_tests[] = {
> > >  		.func = test__parse_events,
> > >  	},
> > >  	{
> > > +		.desc = "Simple expression parser",
> > > +		.func = test__expr,
> > > +	},
> > > +	{
> > >  		.desc = "PERF_RECORD_* events & perf_sample fields",
> > >  		.func = test__PERF_RECORD,
> > >  	},
> > > diff --git a/tools/perf/tests/expr.c b/tools/perf/tests/expr.c
> > > new file mode 100644
> > > index 000000000000..554695c06c5b
> > > --- /dev/null
> > > +++ b/tools/perf/tests/expr.c
> > > @@ -0,0 +1,55 @@
> > > +#include "util/debug.h"
> > > +#include "util/expr.h"
> > > +#include "tests.h"
> > > +
> > > +static int test(struct parse_ctx *ctx, const char *e, double val2)
> > > +{
> > > +	double val;
> > > +
> > > +	if (expr__parse(&val, ctx, &e))
> > > +		TEST_ASSERT_VAL("parse test failed", 0);
> > > +	TEST_ASSERT_VAL("unexpected value", val == val2);
> > > +	return 0;
> > > +}
> > > +
> > > +int test__expr(int subtest __maybe_unused)
> > > +{
> > > +	const char *p;
> > > +	const char **other;
> > > +	double val;
> > > +	int ret;
> > > +	struct parse_ctx ctx;
> > > +	int num_other;
> > > +
> > > +	expr__ctx_init(&ctx);
> > > +	expr__add_id(&ctx, "FOO", 1);
> > > +	expr__add_id(&ctx, "BAR", 2);
> > > +
> > > +	ret = test(&ctx, "1+1", 2);
> > > +	ret |= test(&ctx, "FOO+BAR", 3);
> > > +	ret |= test(&ctx, "(BAR/2)%2", 1);
> > > +	ret |= test(&ctx, "1 - -4",  5);
> > > +	ret |= test(&ctx, "(FOO-1)*2 + (BAR/2)%2 - -4",  5);
> > > +
> > > +	if (ret)
> > > +		return ret;
> > > +
> > > +	p = "FOO/0";
> > > +	ret = expr__parse(&val, &ctx, &p);
> > > +	TEST_ASSERT_VAL("division by zero", ret == 1);
> > > +
> > > +	p = "BAR/";
> > > +	ret = expr__parse(&val, &ctx, &p);
> > > +	TEST_ASSERT_VAL("missing operand", ret == 1);
> > > +
> > > +	TEST_ASSERT_VAL("find other",
> > > +			expr__find_other("FOO + BAR + BAZ + BOZO", "FOO", &other, &num_other) == 0);
> > > +	TEST_ASSERT_VAL("find other", num_other == 3);
> > > +	TEST_ASSERT_VAL("find other", !strcmp(other[0], "BAR"));
> > > +	TEST_ASSERT_VAL("find other", !strcmp(other[1], "BAZ"));
> > > +	TEST_ASSERT_VAL("find other", !strcmp(other[2], "BOZO"));
> > > +	TEST_ASSERT_VAL("find other", other[3] == NULL);
> > > +	free((void *)other);
> > > +
> > > +	return 0;
> > > +}
> > > diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h
> > > index 1fa9b9d83aa5..631859629403 100644
> > > --- a/tools/perf/tests/tests.h
> > > +++ b/tools/perf/tests/tests.h
> > > @@ -62,6 +62,7 @@ int test__sample_parsing(int subtest);
> > >  int test__keep_tracking(int subtest);
> > >  int test__parse_no_sample_id_all(int subtest);
> > >  int test__dwarf_unwind(int subtest);
> > > +int test__expr(int subtest);
> > >  int test__hists_filter(int subtest);
> > >  int test__mmap_thread_lookup(int subtest);
> > >  int test__thread_mg_share(int subtest);
> > > diff --git a/tools/perf/util/Build b/tools/perf/util/Build
> > > index fb4f42f1bb38..0b98534a9ea1 100644
> > > --- a/tools/perf/util/Build
> > > +++ b/tools/perf/util/Build
> > > @@ -90,6 +90,7 @@ libperf-y += mem-events.o
> > >  libperf-y += vsprintf.o
> > >  libperf-y += drv_configs.o
> > >  libperf-y += time-utils.o
> > > +libperf-y += expr-bison.o
> > >  
> > >  libperf-$(CONFIG_LIBBPF) += bpf-loader.o
> > >  libperf-$(CONFIG_BPF_PROLOGUE) += bpf-prologue.o
> > > @@ -142,6 +143,10 @@ $(OUTPUT)util/parse-events-bison.c: util/parse-events.y
> > >  	$(call rule_mkdir)
> > >  	$(Q)$(call echo-cmd,bison)$(BISON) -v util/parse-events.y -d $(PARSER_DEBUG_BISON) -o $@ -p parse_events_
> > >  
> > > +$(OUTPUT)util/expr-bison.c: util/expr.y
> > > +	$(call rule_mkdir)
> > > +	$(Q)$(call echo-cmd,bison)$(BISON) -v util/expr.y -d $(PARSER_DEBUG_BISON) -o $@ -p expr__
> > > +
> > >  $(OUTPUT)util/pmu-flex.c: util/pmu.l $(OUTPUT)util/pmu-bison.c
> > >  	$(call rule_mkdir)
> > >  	$(Q)$(call echo-cmd,flex)$(FLEX) -o $@ --header-file=$(OUTPUT)util/pmu-flex.h util/pmu.l
> > > diff --git a/tools/perf/util/expr.h b/tools/perf/util/expr.h
> > > new file mode 100644
> > > index 000000000000..9c2760a1a96e
> > > --- /dev/null
> > > +++ b/tools/perf/util/expr.h
> > > @@ -0,0 +1,25 @@
> > > +#ifndef PARSE_CTX_H
> > > +#define PARSE_CTX_H 1
> > > +
> > > +#define EXPR_MAX_OTHER 8
> > > +#define MAX_PARSE_ID EXPR_MAX_OTHER
> > > +
> > > +struct parse_id {
> > > +	const char *name;
> > > +	double val;
> > > +};
> > > +
> > > +struct parse_ctx {
> > > +	int num_ids;
> > > +	struct parse_id ids[MAX_PARSE_ID];
> > > +};
> > > +
> > > +void expr__ctx_init(struct parse_ctx *ctx);
> > > +void expr__add_id(struct parse_ctx *ctx, const char *id, double val);
> > > +#ifndef IN_EXPR_Y
> > > +int expr__parse(double *final_val, struct parse_ctx *ctx, const char **pp);
> > > +#endif
> > > +int expr__find_other(const char *p, const char *one, const char ***other,
> > > +		int *num_other);
> > > +
> > > +#endif
> > > diff --git a/tools/perf/util/expr.y b/tools/perf/util/expr.y
> > > new file mode 100644
> > > index 000000000000..a8daa7a93605
> > > --- /dev/null
> > > +++ b/tools/perf/util/expr.y
> > > @@ -0,0 +1,173 @@
> > > +/* Simple expression parser */
> > > +%{
> > > +#include "util.h"
> > > +#include "util/debug.h"
> > > +#define IN_EXPR_Y 1
> > > +#include "expr.h"
> > > +#include <string.h>
> > > +
> > > +#define MAXIDLEN 256
> > > +%}
> > > +
> > > +%define api.pure full
> > > +%parse-param { double *final_val }
> > > +%parse-param { struct parse_ctx *ctx }
> > > +%parse-param { const char **pp }
> > > +%lex-param { const char **pp }
> > > +
> > > +%union {
> > > +	double num;
> > > +	char id[MAXIDLEN+1];
> > > +}
> > > +
> > > +%token <num> NUMBER
> > > +%token <id> ID
> > > +%left '|'
> > > +%left '^'
> > > +%left '&'
> > > +%left '-' '+'
> > > +%left '*' '/' '%'
> > > +%left NEG NOT
> > > +%type <num> expr
> > > +
> > > +%{
> > > +static int expr__lex(YYSTYPE *res, const char **pp);
> > > +
> > > +static void expr__error(double *final_val __maybe_unused,
> > > +		       struct parse_ctx *ctx __maybe_unused,
> > > +		       const char **pp __maybe_unused,
> > > +		       const char *s)
> > > +{
> > > +	pr_debug("%s\n", s);
> > > +}
> > > +
> > > +static int lookup_id(struct parse_ctx *ctx, char *id, double *val)
> > > +{
> > > +	int i;
> > > +
> > > +	for (i = 0; i < ctx->num_ids; i++) {
> > > +		if (!strcasecmp(ctx->ids[i].name, id)) {
> > > +			*val = ctx->ids[i].val;
> > > +			return 0;
> > > +		}
> > > +	}
> > > +	return -1;
> > > +}
> > > +
> > > +%}
> > > +%%
> > > +
> > > +all_expr: expr			{ *final_val = $1; }
> > > +	;
> > > +
> > > +expr:	  NUMBER
> > > +	| ID			{ if (lookup_id(ctx, $1, &$$) < 0) {
> > > +					pr_debug("%s not found", $1);
> > > +					YYABORT;
> > > +				  }
> > > +				}
> > > +	| expr '+' expr		{ $$ = $1 + $3; }
> > > +	| expr '-' expr		{ $$ = $1 - $3; }
> > > +	| expr '*' expr		{ $$ = $1 * $3; }
> > > +	| expr '/' expr		{ if ($3 == 0) YYABORT; $$ = $1 / $3; }
> > > +	| expr '%' expr		{ if ((long)$3 == 0) YYABORT; $$ = (long)$1 % (long)$3; }
> > > +	| '-' expr %prec NEG	{ $$ = -$2; }
> > > +	| '(' expr ')'		{ $$ = $2; }
> > > +	;
> > > +
> > > +%%
> > > +
> > > +static int expr__symbol(YYSTYPE *res, const char *p, const char **pp)
> > > +{
> > > +	char *dst = res->id;
> > > +	const char *s = p;
> > > +
> > > +	while (isalnum(*p) || *p == '_' || *p == '.') {
> > > +		if (p - s >= MAXIDLEN)
> > > +			return -1;
> > > +		*dst++ = *p++;
> > > +	}
> > > +	*dst = 0;
> > > +	*pp = p;
> > > +	return ID;
> > > +}
> > > +
> > > +static int expr__lex(YYSTYPE *res, const char **pp)
> > > +{
> > > +	int tok;
> > > +	const char *s;
> > > +	const char *p = *pp;
> > > +
> > > +	while (isspace(*p))
> > > +		p++;
> > > +	s = p;
> > > +	switch (*p++) {
> > > +	case 'a' ... 'z':
> > > +	case 'A' ... 'Z':
> > > +		return expr__symbol(res, p - 1, pp);
> > > +	case '0' ... '9': case '.':
> > > +		res->num = strtod(s, (char **)&p);
> > > +		tok = NUMBER;
> > > +		break;
> > > +	default:
> > > +		tok = *s;
> > > +		break;
> > > +	}
> > > +	*pp = p;
> > > +	return tok;
> > > +}
> > > +
> > > +/* Caller must make sure id is allocated */
> > > +void expr__add_id(struct parse_ctx *ctx, const char *name, double val)
> > > +{
> > > +	int idx;
> > > +	assert(ctx->num_ids < MAX_PARSE_ID);
> > > +	idx = ctx->num_ids++;
> > > +	ctx->ids[idx].name = name;
> > > +	ctx->ids[idx].val = val;
> > > +}
> > > +
> > > +void expr__ctx_init(struct parse_ctx *ctx)
> > > +{
> > > +	ctx->num_ids = 0;
> > > +}
> > > +
> > > +int expr__find_other(const char *p, const char *one, const char ***other,
> > > +		     int *num_otherp)
> > > +{
> > > +	const char *orig = p;
> > > +	int err = -1;
> > > +	int num_other;
> > > +
> > > +	*other = malloc((EXPR_MAX_OTHER + 1) * sizeof(char *));
> > > +	if (!*other)
> > > +		return -1;
> > > +
> > > +	num_other = 0;
> > > +	for (;;) {
> > > +		YYSTYPE val;
> > > +		int tok = expr__lex(&val, &p);
> > > +		if (tok == 0) {
> > > +			err = 0;
> > > +			break;
> > > +		}
> > > +		if (tok == ID && strcasecmp(one, val.id)) {
> > > +			if (num_other >= EXPR_MAX_OTHER - 1) {
> > > +				pr_debug("Too many extra events in %s\n", orig);
> > > +				break;
> > > +			}
> > > +			(*other)[num_other] = strdup(val.id);
> > > +			if (!(*other)[num_other])
> > > +				return -1;
> > > +			num_other++;
> > > +		}
> > > +	}
> > > +	(*other)[num_other] = NULL;
> > > +	*num_otherp = num_other;
> > > +	if (err) {
> > > +		*num_otherp = 0;
> > > +		free(*other);
> > > +		*other = NULL;
> > > +	}
> > > +	return err;
> > > +}
> > > -- 
> > > 2.9.3

[toc] | [prev] | [next] | [standalone]

#1608874 — [tip:perf/core] perf tools: Add a simple expression parser for JSON

Fromtip-bot for Andi Kleen <tipbot@zytor.com>
Date2017-03-24 20:00 +0100
Subject[tip:perf/core] perf tools: Add a simple expression parser for JSON
Message-ID<toCBj-5D8-1@gated-at.bofh.it>
In reply to#1605122
Commit-ID:  075167363f8b53ade702cd83f5818eb47119b659
Gitweb:     http://git.kernel.org/tip/075167363f8b53ade702cd83f5818eb47119b659
Author:     Andi Kleen <ak@linux.intel.com>
AuthorDate: Mon, 20 Mar 2017 13:17:05 -0700
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Thu, 23 Mar 2017 11:39:27 -0300

perf tools: Add a simple expression parser for JSON

Add a simple expression parser good enough to parse JSON relation
expressions. The parser is implemented using bison.

This is just intended as an simple parser for internal usage in the
event lists, not the beginning of a "perf scripting language"

v2: Use expr__ prefix instead of expr_
    Support multiple free variables for parser

Committer note:

The v2 patch had:

  %define api.pure full

In expr.y, that is a feature introduced in bison 2.7, to have reentrant
parsers, not using global variables, which would make tools/perf stop
building with the bison version shipped in older distros, so Andi
realised that the other parsers (e.g. parse-events.y) were using:

  %pure-parser

Which is present in older versions of bison and fits the bill.

I added:

  CFLAGS_expr-bison.o += -DYYENABLE_NLS=0 -DYYLTYPE_IS_TRIVIAL=0 -w

To finally make it build, copying what was there for pmu-bison.o,
another parser.

Signed-off-by: Andi Kleen <ak@linux.intel.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Link: http://lkml.kernel.org/r/20170320201711.14142-8-andi@firstfloor.org
[ stdlib.h is needed in tests/expr.c for free() fixing build in systems such as ubuntu:16.04-x-s390 ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/tests/Build          |   1 +
 tools/perf/tests/builtin-test.c |   4 +
 tools/perf/tests/expr.c         |  56 +++++++++++++
 tools/perf/tests/tests.h        |   1 +
 tools/perf/util/Build           |   6 ++
 tools/perf/util/expr.h          |  25 ++++++
 tools/perf/util/expr.y          | 173 ++++++++++++++++++++++++++++++++++++++++
 7 files changed, 266 insertions(+)

diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
index 1cb3d9b..af58ebc 100644
--- a/tools/perf/tests/Build
+++ b/tools/perf/tests/Build
@@ -38,6 +38,7 @@ perf-y += cpumap.o
 perf-y += stat.o
 perf-y += event_update.o
 perf-y += event-times.o
+perf-y += expr.o
 perf-y += backward-ring-buffer.o
 perf-y += sdt.o
 perf-y += is_printable_array.o
diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c
index 83c4669..8682296 100644
--- a/tools/perf/tests/builtin-test.c
+++ b/tools/perf/tests/builtin-test.c
@@ -44,6 +44,10 @@ static struct test generic_tests[] = {
 		.func = test__parse_events,
 	},
 	{
+		.desc = "Simple expression parser",
+		.func = test__expr,
+	},
+	{
 		.desc = "PERF_RECORD_* events & perf_sample fields",
 		.func = test__PERF_RECORD,
 	},
diff --git a/tools/perf/tests/expr.c b/tools/perf/tests/expr.c
new file mode 100644
index 0000000..6c6a374
--- /dev/null
+++ b/tools/perf/tests/expr.c
@@ -0,0 +1,56 @@
+#include "util/debug.h"
+#include "util/expr.h"
+#include "tests.h"
+#include <stdlib.h>
+
+static int test(struct parse_ctx *ctx, const char *e, double val2)
+{
+	double val;
+
+	if (expr__parse(&val, ctx, &e))
+		TEST_ASSERT_VAL("parse test failed", 0);
+	TEST_ASSERT_VAL("unexpected value", val == val2);
+	return 0;
+}
+
+int test__expr(int subtest __maybe_unused)
+{
+	const char *p;
+	const char **other;
+	double val;
+	int ret;
+	struct parse_ctx ctx;
+	int num_other;
+
+	expr__ctx_init(&ctx);
+	expr__add_id(&ctx, "FOO", 1);
+	expr__add_id(&ctx, "BAR", 2);
+
+	ret = test(&ctx, "1+1", 2);
+	ret |= test(&ctx, "FOO+BAR", 3);
+	ret |= test(&ctx, "(BAR/2)%2", 1);
+	ret |= test(&ctx, "1 - -4",  5);
+	ret |= test(&ctx, "(FOO-1)*2 + (BAR/2)%2 - -4",  5);
+
+	if (ret)
+		return ret;
+
+	p = "FOO/0";
+	ret = expr__parse(&val, &ctx, &p);
+	TEST_ASSERT_VAL("division by zero", ret == 1);
+
+	p = "BAR/";
+	ret = expr__parse(&val, &ctx, &p);
+	TEST_ASSERT_VAL("missing operand", ret == 1);
+
+	TEST_ASSERT_VAL("find other",
+			expr__find_other("FOO + BAR + BAZ + BOZO", "FOO", &other, &num_other) == 0);
+	TEST_ASSERT_VAL("find other", num_other == 3);
+	TEST_ASSERT_VAL("find other", !strcmp(other[0], "BAR"));
+	TEST_ASSERT_VAL("find other", !strcmp(other[1], "BAZ"));
+	TEST_ASSERT_VAL("find other", !strcmp(other[2], "BOZO"));
+	TEST_ASSERT_VAL("find other", other[3] == NULL);
+	free((void *)other);
+
+	return 0;
+}
diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h
index 1fa9b9d..6318596 100644
--- a/tools/perf/tests/tests.h
+++ b/tools/perf/tests/tests.h
@@ -62,6 +62,7 @@ int test__sample_parsing(int subtest);
 int test__keep_tracking(int subtest);
 int test__parse_no_sample_id_all(int subtest);
 int test__dwarf_unwind(int subtest);
+int test__expr(int subtest);
 int test__hists_filter(int subtest);
 int test__mmap_thread_lookup(int subtest);
 int test__thread_mg_share(int subtest);
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index fb4f42f..2ae92da 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -90,6 +90,7 @@ libperf-y += mem-events.o
 libperf-y += vsprintf.o
 libperf-y += drv_configs.o
 libperf-y += time-utils.o
+libperf-y += expr-bison.o
 
 libperf-$(CONFIG_LIBBPF) += bpf-loader.o
 libperf-$(CONFIG_BPF_PROLOGUE) += bpf-prologue.o
@@ -142,6 +143,10 @@ $(OUTPUT)util/parse-events-bison.c: util/parse-events.y
 	$(call rule_mkdir)
 	$(Q)$(call echo-cmd,bison)$(BISON) -v util/parse-events.y -d $(PARSER_DEBUG_BISON) -o $@ -p parse_events_
 
+$(OUTPUT)util/expr-bison.c: util/expr.y
+	$(call rule_mkdir)
+	$(Q)$(call echo-cmd,bison)$(BISON) -v util/expr.y -d $(PARSER_DEBUG_BISON) -o $@ -p expr__
+
 $(OUTPUT)util/pmu-flex.c: util/pmu.l $(OUTPUT)util/pmu-bison.c
 	$(call rule_mkdir)
 	$(Q)$(call echo-cmd,flex)$(FLEX) -o $@ --header-file=$(OUTPUT)util/pmu-flex.h util/pmu.l
@@ -154,6 +159,7 @@ CFLAGS_parse-events-flex.o  += -w
 CFLAGS_pmu-flex.o           += -w
 CFLAGS_parse-events-bison.o += -DYYENABLE_NLS=0 -w
 CFLAGS_pmu-bison.o          += -DYYENABLE_NLS=0 -DYYLTYPE_IS_TRIVIAL=0 -w
+CFLAGS_expr-bison.o         += -DYYENABLE_NLS=0 -DYYLTYPE_IS_TRIVIAL=0 -w
 
 $(OUTPUT)util/parse-events.o: $(OUTPUT)util/parse-events-flex.c $(OUTPUT)util/parse-events-bison.c
 $(OUTPUT)util/pmu.o: $(OUTPUT)util/pmu-flex.c $(OUTPUT)util/pmu-bison.c
diff --git a/tools/perf/util/expr.h b/tools/perf/util/expr.h
new file mode 100644
index 0000000..9c2760a
--- /dev/null
+++ b/tools/perf/util/expr.h
@@ -0,0 +1,25 @@
+#ifndef PARSE_CTX_H
+#define PARSE_CTX_H 1
+
+#define EXPR_MAX_OTHER 8
+#define MAX_PARSE_ID EXPR_MAX_OTHER
+
+struct parse_id {
+	const char *name;
+	double val;
+};
+
+struct parse_ctx {
+	int num_ids;
+	struct parse_id ids[MAX_PARSE_ID];
+};
+
+void expr__ctx_init(struct parse_ctx *ctx);
+void expr__add_id(struct parse_ctx *ctx, const char *id, double val);
+#ifndef IN_EXPR_Y
+int expr__parse(double *final_val, struct parse_ctx *ctx, const char **pp);
+#endif
+int expr__find_other(const char *p, const char *one, const char ***other,
+		int *num_other);
+
+#endif
diff --git a/tools/perf/util/expr.y b/tools/perf/util/expr.y
new file mode 100644
index 0000000..954556b
--- /dev/null
+++ b/tools/perf/util/expr.y
@@ -0,0 +1,173 @@
+/* Simple expression parser */
+%{
+#include "util.h"
+#include "util/debug.h"
+#define IN_EXPR_Y 1
+#include "expr.h"
+#include <string.h>
+
+#define MAXIDLEN 256
+%}
+
+%pure-parser
+%parse-param { double *final_val }
+%parse-param { struct parse_ctx *ctx }
+%parse-param { const char **pp }
+%lex-param { const char **pp }
+
+%union {
+	double num;
+	char id[MAXIDLEN+1];
+}
+
+%token <num> NUMBER
+%token <id> ID
+%left '|'
+%left '^'
+%left '&'
+%left '-' '+'
+%left '*' '/' '%'
+%left NEG NOT
+%type <num> expr
+
+%{
+static int expr__lex(YYSTYPE *res, const char **pp);
+
+static void expr__error(double *final_val __maybe_unused,
+		       struct parse_ctx *ctx __maybe_unused,
+		       const char **pp __maybe_unused,
+		       const char *s)
+{
+	pr_debug("%s\n", s);
+}
+
+static int lookup_id(struct parse_ctx *ctx, char *id, double *val)
+{
+	int i;
+
+	for (i = 0; i < ctx->num_ids; i++) {
+		if (!strcasecmp(ctx->ids[i].name, id)) {
+			*val = ctx->ids[i].val;
+			return 0;
+		}
+	}
+	return -1;
+}
+
+%}
+%%
+
+all_expr: expr			{ *final_val = $1; }
+	;
+
+expr:	  NUMBER
+	| ID			{ if (lookup_id(ctx, $1, &$$) < 0) {
+					pr_debug("%s not found", $1);
+					YYABORT;
+				  }
+				}
+	| expr '+' expr		{ $$ = $1 + $3; }
+	| expr '-' expr		{ $$ = $1 - $3; }
+	| expr '*' expr		{ $$ = $1 * $3; }
+	| expr '/' expr		{ if ($3 == 0) YYABORT; $$ = $1 / $3; }
+	| expr '%' expr		{ if ((long)$3 == 0) YYABORT; $$ = (long)$1 % (long)$3; }
+	| '-' expr %prec NEG	{ $$ = -$2; }
+	| '(' expr ')'		{ $$ = $2; }
+	;
+
+%%
+
+static int expr__symbol(YYSTYPE *res, const char *p, const char **pp)
+{
+	char *dst = res->id;
+	const char *s = p;
+
+	while (isalnum(*p) || *p == '_' || *p == '.') {
+		if (p - s >= MAXIDLEN)
+			return -1;
+		*dst++ = *p++;
+	}
+	*dst = 0;
+	*pp = p;
+	return ID;
+}
+
+static int expr__lex(YYSTYPE *res, const char **pp)
+{
+	int tok;
+	const char *s;
+	const char *p = *pp;
+
+	while (isspace(*p))
+		p++;
+	s = p;
+	switch (*p++) {
+	case 'a' ... 'z':
+	case 'A' ... 'Z':
+		return expr__symbol(res, p - 1, pp);
+	case '0' ... '9': case '.':
+		res->num = strtod(s, (char **)&p);
+		tok = NUMBER;
+		break;
+	default:
+		tok = *s;
+		break;
+	}
+	*pp = p;
+	return tok;
+}
+
+/* Caller must make sure id is allocated */
+void expr__add_id(struct parse_ctx *ctx, const char *name, double val)
+{
+	int idx;
+	assert(ctx->num_ids < MAX_PARSE_ID);
+	idx = ctx->num_ids++;
+	ctx->ids[idx].name = name;
+	ctx->ids[idx].val = val;
+}
+
+void expr__ctx_init(struct parse_ctx *ctx)
+{
+	ctx->num_ids = 0;
+}
+
+int expr__find_other(const char *p, const char *one, const char ***other,
+		     int *num_otherp)
+{
+	const char *orig = p;
+	int err = -1;
+	int num_other;
+
+	*other = malloc((EXPR_MAX_OTHER + 1) * sizeof(char *));
+	if (!*other)
+		return -1;
+
+	num_other = 0;
+	for (;;) {
+		YYSTYPE val;
+		int tok = expr__lex(&val, &p);
+		if (tok == 0) {
+			err = 0;
+			break;
+		}
+		if (tok == ID && strcasecmp(one, val.id)) {
+			if (num_other >= EXPR_MAX_OTHER - 1) {
+				pr_debug("Too many extra events in %s\n", orig);
+				break;
+			}
+			(*other)[num_other] = strdup(val.id);
+			if (!(*other)[num_other])
+				return -1;
+			num_other++;
+		}
+	}
+	(*other)[num_other] = NULL;
+	*num_otherp = num_other;
+	if (err) {
+		*num_otherp = 0;
+		free(*other);
+		*other = NULL;
+	}
+	return err;
+}

[toc] | [prev] | [next] | [standalone]

#1605671

FromJiri Olsa <jolsa@redhat.com>
Date2017-03-21 15:50 +0100
Message-ID<tntgJ-4Dp-17@gated-at.bofh.it>
In reply to#1605110
On Mon, Mar 20, 2017 at 01:16:58PM -0700, Andi Kleen wrote:
> This patch kit further improves support for Intel uncore events in
> the Linux perf user tool. The basic support has been already
> merged earlier, but this makes it nicer to use.
> 
> - Collapse counts from duplicated boxes to make the output
> easier to read.
> - Support specifying events for multiple duplicated boxes
> in an abbreviated format to shorten event specifiers
> - Add support for computing Metrics defined in the event lists,
> so that the event lists can extend the metrics in perf stat.
> This allows to represent many events in an easier to understand
> format.
> 
> Available from
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-misc.git perf/builtin-json-30
> 
> v1: Initial post after being split off to own patchkit
> Adding MetricName support and support for more than two events
> in expressions.
> 
> v2: Address review comments. Move new hunk from refactor
> patchkit to patch adding new features. Improve changelogs
> slightly.
> 
> v3: Rebased on latest tip tree.
> Split some patches based on review feedback.
> Add event list changes for MetricName
> Move extra printing in perf list to new option
> Improve descriptions
> Cleanups based on code review.
> 
> v4: Use perf_evsel__cpus in refactoring
> Avoid multiple warning messages for missing metric events.
> Rebase on latest tip

except for the nit in one changelog, for the patchset:

Acked-by: Jiri Olsa <jolsa@kernel.org>

thanks,
jirka

[toc] | [prev] | [next] | [standalone]

#1605961

FromArnaldo Carvalho de Melo <acme@kernel.org>
Date2017-03-21 21:00 +0100
Message-ID<tny6K-7WE-9@gated-at.bofh.it>
In reply to#1605671
Em Tue, Mar 21, 2017 at 03:48:40PM +0100, Jiri Olsa escreveu:
> On Mon, Mar 20, 2017 at 01:16:58PM -0700, Andi Kleen wrote:
> > v4: Use perf_evsel__cpus in refactoring
> > Avoid multiple warning messages for missing metric events.
> > Rebase on latest tip
 
> except for the nit in one changelog, for the patchset:
 
> Acked-by: Jiri Olsa <jolsa@kernel.org>

Thanks, applied to perf/core.

- Arnaldo

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | linux.kernel

csiph-web