Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1174889 > unrolled thread

[RFC PATCH v10 00/50] perf tools: filtering events using eBPF programs

Back to article view | Back to linux.kernel

Contents

  [RFC PATCH v10 00/50] perf tools: filtering events using eBPF programs Wang Nan <wangnan0@huawei.com> - 2015-07-01 04:20 +0200
    [RFC PATCH v10 18/50] bpf tools: Introduce bpf_load_program() to bpf.c Wang Nan <wangnan0@huawei.com> - 2015-07-01 04:20 +0200
    [RFC PATCH v10 07/50] bpf tools: Check endianness and make libbpf fail early Wang Nan <wangnan0@huawei.com> - 2015-07-01 04:20 +0200
    [RFC PATCH v10 25/50] perf tools: Call clang to compile C source to object code Wang Nan <wangnan0@huawei.com> - 2015-07-01 04:20 +0200
    [RFC PATCH v10 02/50] tracing, perf: Implement BPF programs attached to uprobes Wang Nan <wangnan0@huawei.com> - 2015-07-01 04:20 +0200
    [RFC PATCH v10 05/50] bpf tools: Open eBPF object file and do basic validation Wang Nan <wangnan0@huawei.com> - 2015-07-01 04:20 +0200
    [RFC PATCH v10 22/50] bpf tools: Link all bpf objects onto a list Wang Nan <wangnan0@huawei.com> - 2015-07-01 04:20 +0200
    [RFC PATCH v10 27/50] perf tools: Auto detecting kernel build directory Wang Nan <wangnan0@huawei.com> - 2015-07-01 04:20 +0200
    [RFC PATCH v10 06/50] bpf tools: Read eBPF object from buffer Wang Nan <wangnan0@huawei.com> - 2015-07-01 04:20 +0200
    [RFC PATCH v10 13/50] bpf tools: Collect relocation sections from SHT_REL sections Wang Nan <wangnan0@huawei.com> - 2015-07-01 04:20 +0200
    [RFC PATCH v10 01/50] bpf: Use correct #ifdef controller for trace_call_bpf() Wang Nan <wangnan0@huawei.com> - 2015-07-01 04:20 +0200
    [RFC PATCH v10 16/50] bpf tools: Create eBPF maps defined in an object file Wang Nan <wangnan0@huawei.com> - 2015-07-01 04:20 +0200
    [RFC PATCH v10 17/50] bpf tools: Relocate eBPF programs Wang Nan <wangnan0@huawei.com> - 2015-07-01 04:20 +0200
    Re: [RFC PATCH v10 23/50] perf tools: Make perf depend on libbpf Arnaldo Carvalho de Melo <acme@kernel.org> - 2015-07-08 15:10 +0200
    Re: [RFC PATCH v10 23/50] perf tools: Make perf depend on libbpf Arnaldo Carvalho de Melo <acme@kernel.org> - 2015-07-08 15:10 +0200

#1174889 — [RFC PATCH v10 00/50] perf tools: filtering events using eBPF programs

This is the 10th version which tries to introduce eBPF programs to perf.

Compare with v9 there are only some small changes:

 1. Squash patch 2/50 and 3/50 in v9 together, never check host
    include/linux/bpf.h, so bpf feature check will always green.

 2. Fix compiling error when CONFIG_BPF_SYSCALL is set but
    CONFIG_BPF_EVENT is unset. See patch 1/50.

 3. In add_perf_probe_events(), always init kernel symbol maps.

He Kuang (3):
  perf tools: Move linux/filter.h to tools/include
  perf tools: Introduce arch_get_reg_info() for x86
  perf record: Support custom vmlinux path

Wang Nan (47):
  bpf: Use correct #ifdef controller for trace_call_bpf()
  tracing, perf: Implement BPF programs attached to uprobes
  bpf tools: Introduce 'bpf' library and add bpf feature check
  bpf tools: Allow caller to set printing function
  bpf tools: Open eBPF object file and do basic validation
  bpf tools: Read eBPF object from buffer
  bpf tools: Check endianness and make libbpf fail early
  bpf tools: Iterate over ELF sections to collect information
  bpf tools: Collect version and license from ELF sections
  bpf tools: Collect map definitions from 'maps' section
  bpf tools: Collect symbol table from SHT_SYMTAB section
  bpf tools: Collect eBPF programs from their own sections
  bpf tools: Collect relocation sections from SHT_REL sections
  bpf tools: Record map accessing instructions for each program
  bpf tools: Add bpf.c/h for common bpf operations
  bpf tools: Create eBPF maps defined in an object file
  bpf tools: Relocate eBPF programs
  bpf tools: Introduce bpf_load_program() to bpf.c
  bpf tools: Load eBPF programs in object files into kernel
  bpf tools: Introduce accessors for struct bpf_program
  bpf tools: Introduce accessors for struct bpf_object
  bpf tools: Link all bpf objects onto a list
  perf tools: Make perf depend on libbpf
  perf tools: Introduce llvm config options
  perf tools: Call clang to compile C source to object code
  perf tests: Add LLVM test for eBPF on-the-fly compiling
  perf tools: Auto detecting kernel build directory
  perf tools: Auto detecting kernel include options
  perf record: Enable passing bpf object file to --event
  perf record: Compile scriptlets if pass '.c' to --event
  perf tools: Parse probe points of eBPF programs during preparation
  perf probe: Attach trace_probe_event with perf_probe_event
  perf record: Probe at kprobe points
  perf record: Load all eBPF object into kernel
  perf tools: Add bpf_fd field to evsel and config it
  perf tools: Attach eBPF program to perf event
  perf tools: Suppress probing messages when probing by BPF loading
  perf record: Add clang options for compiling BPF scripts
  bpf tools: Load a program with different instance using preprocessor
  perf tools: Fix probe-event.h include
  perf probe: Reset tev->args and tev->nargs when failure
  perf tools: Add BPF_PROLOGUE config options for further patches
  perf tools: Add prologue for BPF programs for fetching arguments
  perf tools: Generate prologue for BPF programs
  perf tools: Use same BPF program if arguments are identical
  perf probe: Init symbol as kprobe if any event is kprobe
  perf tools: Support attach BPF program on uprobe events

 include/linux/ftrace_event.h          |    7 +-
 kernel/events/core.c                  |    4 +-
 kernel/trace/Kconfig                  |    2 +-
 kernel/trace/trace_uprobe.c           |    5 +
 tools/build/Makefile.feature          |    6 +-
 tools/build/feature/Makefile          |    6 +-
 tools/build/feature/test-bpf.c        |   18 +
 tools/include/linux/filter.h          |  237 +++++++
 tools/lib/bpf/.gitignore              |    2 +
 tools/lib/bpf/Build                   |    1 +
 tools/lib/bpf/Makefile                |  195 ++++++
 tools/lib/bpf/bpf.c                   |   85 +++
 tools/lib/bpf/bpf.h                   |   23 +
 tools/lib/bpf/libbpf.c                | 1174 +++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf.h                |  107 +++
 tools/perf/MANIFEST                   |    4 +
 tools/perf/Makefile.perf              |   19 +-
 tools/perf/arch/x86/Makefile          |    1 +
 tools/perf/arch/x86/util/Build        |    2 +
 tools/perf/arch/x86/util/dwarf-regs.c |  104 ++-
 tools/perf/builtin-probe.c            |    4 +-
 tools/perf/builtin-record.c           |   47 +-
 tools/perf/config/Makefile            |   31 +-
 tools/perf/tests/Build                |    1 +
 tools/perf/tests/builtin-test.c       |    4 +
 tools/perf/tests/llvm.c               |   85 +++
 tools/perf/tests/make                 |    4 +-
 tools/perf/tests/tests.h              |    1 +
 tools/perf/util/Build                 |    3 +
 tools/perf/util/bpf-loader.c          |  631 ++++++++++++++++++
 tools/perf/util/bpf-loader.h          |   46 ++
 tools/perf/util/bpf-prologue.c        |  442 +++++++++++++
 tools/perf/util/bpf-prologue.h        |   34 +
 tools/perf/util/config.c              |    4 +
 tools/perf/util/debug.c               |    5 +
 tools/perf/util/debug.h               |    1 +
 tools/perf/util/evlist.c              |   41 ++
 tools/perf/util/evlist.h              |    1 +
 tools/perf/util/evsel.c               |   17 +
 tools/perf/util/evsel.h               |    1 +
 tools/perf/util/include/dwarf-regs.h  |    7 +
 tools/perf/util/llvm-utils.c          |  370 +++++++++++
 tools/perf/util/llvm-utils.h          |   39 ++
 tools/perf/util/parse-events.c        |   16 +
 tools/perf/util/parse-events.h        |    2 +
 tools/perf/util/parse-events.l        |    6 +
 tools/perf/util/parse-events.y        |   29 +-
 tools/perf/util/probe-event.c         |   84 +--
 tools/perf/util/probe-event.h         |    8 +-
 tools/perf/util/probe-finder.c        |    4 +
 50 files changed, 3889 insertions(+), 81 deletions(-)
 create mode 100644 tools/build/feature/test-bpf.c
 create mode 100644 tools/include/linux/filter.h
 create mode 100644 tools/lib/bpf/.gitignore
 create mode 100644 tools/lib/bpf/Build
 create mode 100644 tools/lib/bpf/Makefile
 create mode 100644 tools/lib/bpf/bpf.c
 create mode 100644 tools/lib/bpf/bpf.h
 create mode 100644 tools/lib/bpf/libbpf.c
 create mode 100644 tools/lib/bpf/libbpf.h
 create mode 100644 tools/perf/tests/llvm.c
 create mode 100644 tools/perf/util/bpf-loader.c
 create mode 100644 tools/perf/util/bpf-loader.h
 create mode 100644 tools/perf/util/bpf-prologue.c
 create mode 100644 tools/perf/util/bpf-prologue.h
 create mode 100644 tools/perf/util/llvm-utils.c
 create mode 100644 tools/perf/util/llvm-utils.h

-- 
1.8.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1174890 — [RFC PATCH v10 18/50] bpf tools: Introduce bpf_load_program() to bpf.c

bpf_load_program() can be used to load bpf program into kernel. To make
loading faster, first try to load without logbuf. Try again with logbuf
if the first try failed.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Alexei Starovoitov <ast@plumgrid.com>
---
 tools/lib/bpf/bpf.c | 34 ++++++++++++++++++++++++++++++++++
 tools/lib/bpf/bpf.h |  7 +++++++
 2 files changed, 41 insertions(+)

diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c
index 208de7c3..a633105 100644
--- a/tools/lib/bpf/bpf.c
+++ b/tools/lib/bpf/bpf.c
@@ -29,6 +29,11 @@
 # endif
 #endif
 
+static __u64 ptr_to_u64(void *ptr)
+{
+	return (__u64) (unsigned long) ptr;
+}
+
 static int sys_bpf(enum bpf_cmd cmd, union bpf_attr *attr,
 		   unsigned int size)
 {
@@ -49,3 +54,32 @@ int bpf_create_map(enum bpf_map_type map_type, int key_size,
 
 	return sys_bpf(BPF_MAP_CREATE, &attr, sizeof(attr));
 }
+
+int bpf_load_program(enum bpf_prog_type type, struct bpf_insn *insns,
+		     size_t insns_cnt, char *license,
+		     u32 kern_version, char *log_buf, size_t log_buf_sz)
+{
+	int fd;
+	union bpf_attr attr;
+
+	bzero(&attr, sizeof(attr));
+	attr.prog_type = type;
+	attr.insn_cnt = (__u32)insns_cnt;
+	attr.insns = ptr_to_u64(insns);
+	attr.license = ptr_to_u64(license);
+	attr.log_buf = ptr_to_u64(NULL);
+	attr.log_size = 0;
+	attr.log_level = 0;
+	attr.kern_version = kern_version;
+
+	fd = sys_bpf(BPF_PROG_LOAD, &attr, sizeof(attr));
+	if (fd >= 0 || !log_buf || !log_buf_sz)
+		return fd;
+
+	/* Try again with log */
+	attr.log_buf = ptr_to_u64(log_buf);
+	attr.log_size = log_buf_sz;
+	attr.log_level = 1;
+	log_buf[0] = 0;
+	return sys_bpf(BPF_PROG_LOAD, &attr, sizeof(attr));
+}
diff --git a/tools/lib/bpf/bpf.h b/tools/lib/bpf/bpf.h
index 28f7942..854b736 100644
--- a/tools/lib/bpf/bpf.h
+++ b/tools/lib/bpf/bpf.h
@@ -13,4 +13,11 @@
 int bpf_create_map(enum bpf_map_type map_type, int key_size, int value_size,
 		   int max_entries);
 
+/* Recommend log buffer size */
+#define BPF_LOG_BUF_SIZE 65536
+int bpf_load_program(enum bpf_prog_type type, struct bpf_insn *insns,
+		     size_t insns_cnt, char *license,
+		     u32 kern_version, char *log_buf,
+		     size_t log_buf_sz);
+
 #endif
-- 
1.8.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1174891 — [RFC PATCH v10 07/50] bpf tools: Check endianness and make libbpf fail early

Check endianness according to EHDR. Code is taken from
tools/perf/util/symbol-elf.c.

Libbpf doesn't magically convert missmatched endianness. Even if we swap
eBPF instructions to correct byte order, we are unable to deal with
endianness in code logical generated by LLVM.

Therefore, libbpf should simply reject missmatched ELF object, and let
LLVM to create good code.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Alexei Starovoitov <ast@plumgrid.com>
---
 tools/lib/bpf/libbpf.c | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 36dfbc1..15b3e82 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -192,6 +192,34 @@ errout:
 	return err;
 }
 
+static int
+bpf_object__check_endianness(struct bpf_object *obj)
+{
+	static unsigned int const endian = 1;
+
+	switch (obj->efile.ehdr.e_ident[EI_DATA]) {
+	case ELFDATA2LSB:
+		/* We are big endian, BPF obj is little endian. */
+		if (*(unsigned char const *)&endian != 1)
+			goto mismatch;
+		break;
+
+	case ELFDATA2MSB:
+		/* We are little endian, BPF obj is big endian. */
+		if (*(unsigned char const *)&endian != 0)
+			goto mismatch;
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	return 0;
+
+mismatch:
+	pr_warning("Error: endianness mismatch.\n");
+	return -EINVAL;
+}
+
 static struct bpf_object *
 __bpf_object__open(const char *path, void *obj_buf, size_t obj_buf_sz)
 {
@@ -208,6 +236,8 @@ __bpf_object__open(const char *path, void *obj_buf, size_t obj_buf_sz)
 
 	if (bpf_object__elf_init(obj))
 		goto out;
+	if (bpf_object__check_endianness(obj))
+		goto out;
 
 	bpf_object__elf_finish(obj);
 	return obj;
-- 
1.8.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1174892 — [RFC PATCH v10 25/50] perf tools: Call clang to compile C source to object code

This is the core patch for supporting eBPF on-the-fly compiling, does
the following work:

 1. Search clang compiler using search_program().

 2. Run command template defined in llvm-bpf-cmd-template option in
    [llvm] config section using read_from_pipe(). Patch of clang and
    source code path is injected into shell command using environment
    variable using force_set_env().

Signed-off-by: Wang Nan <wangnan0@huawei.com>
---
 tools/perf/util/llvm-utils.c | 222 +++++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/llvm-utils.h |   3 +
 2 files changed, 225 insertions(+)

diff --git a/tools/perf/util/llvm-utils.c b/tools/perf/util/llvm-utils.c
index fd5b1bc..46d42c3 100644
--- a/tools/perf/util/llvm-utils.c
+++ b/tools/perf/util/llvm-utils.c
@@ -43,3 +43,225 @@ int perf_llvm_config(const char *var, const char *value)
 		return -1;
 	return 0;
 }
+
+static int
+search_program(const char *def, const char *name,
+	       char *output)
+{
+	char *env, *path, *tmp;
+	char buf[PATH_MAX];
+	int ret;
+
+	output[0] = '\0';
+	if (def && def[0] != '\0') {
+		if (def[0] == '/') {
+			if (access(def, F_OK) == 0) {
+				strlcpy(output, def, PATH_MAX);
+				return 0;
+			}
+		} else if (def[0] != '\0')
+			name = def;
+	}
+
+	env = getenv("PATH");
+	if (!env)
+		return -1;
+	env = strdup(env);
+	if (!env)
+		return -1;
+
+	ret = -ENOENT;
+	path = strtok_r(env, ":",  &tmp);
+	while (path) {
+		scnprintf(buf, sizeof(buf), "%s/%s", path, name);
+		if (access(buf, F_OK) == 0) {
+			strlcpy(output, buf, PATH_MAX);
+			ret = 0;
+			break;
+		}
+		path = strtok_r(NULL, ":", &tmp);
+	}
+
+	free(env);
+	return ret;
+}
+
+#define READ_SIZE	4096
+static int
+read_from_pipe(const char *cmd, void **p_buf, size_t *p_read_sz)
+{
+	int err = 0;
+	void *buf = NULL;
+	FILE *file = NULL;
+	size_t read_sz = 0, buf_sz = 0;
+
+	file = popen(cmd, "r");
+	if (!file) {
+		pr_err("ERROR: unable to popen cmd: %s\n",
+		       strerror(errno));
+		return -EINVAL;
+	}
+
+	while (!feof(file) && !ferror(file)) {
+		/*
+		 * Make buf_sz always have obe byte extra space so we
+		 * can put '\0' there.
+		 */
+		if (buf_sz - read_sz < READ_SIZE + 1) {
+			void *new_buf;
+
+			buf_sz = read_sz + READ_SIZE + 1;
+			new_buf = realloc(buf, buf_sz);
+
+			if (!new_buf) {
+				pr_err("ERROR: failed to realloc memory\n");
+				err = -ENOMEM;
+				goto errout;
+			}
+
+			buf = new_buf;
+		}
+		read_sz += fread(buf + read_sz, 1, READ_SIZE, file);
+	}
+
+	if (buf_sz - read_sz < 1) {
+		pr_err("ERROR: internal error\n");
+		err = -EINVAL;
+		goto errout;
+	}
+
+	if (ferror(file)) {
+		pr_err("ERROR: error occurred when reading from pipe: %s\n",
+		       strerror(errno));
+		err = -EIO;
+		goto errout;
+	}
+
+	err = WEXITSTATUS(pclose(file));
+	file = NULL;
+	if (err) {
+		err = -EINVAL;
+		goto errout;
+	}
+
+	/*
+	 * If buf is string, give it terminal '\0' to make our life
+	 * easier. If buf is not string, that '\0' is out of space
+	 * indicated by read_sz so caller won't even notice it.
+	 */
+	((char *)buf)[read_sz] = '\0';
+
+	if (!p_buf)
+		free(buf);
+	else
+		*p_buf = buf;
+
+	if (p_read_sz)
+		*p_read_sz = read_sz;
+	return 0;
+
+errout:
+	if (file)
+		pclose(file);
+	free(buf);
+	if (p_buf)
+		*p_buf = NULL;
+	if (p_read_sz)
+		*p_read_sz = 0;
+	return err;
+}
+
+static inline void
+force_set_env(const char *var, const char *value)
+{
+	if (value) {
+		setenv(var, value, 1);
+		pr_debug("set env: %s=%s\n", var, value);
+	} else {
+		unsetenv(var);
+		pr_debug("unset env: %s\n", var);
+	}
+}
+
+static void
+version_notice(void)
+{
+	pr_err(
+"     \tLLVM 3.7 or newer is required. Which can be found from http://llvm.org\n"
+"     \tYou may want to try git trunk:\n"
+"     \t\tgit clone http://llvm.org/git/llvm.git\n"
+"     \t\t     and\n"
+"     \t\tgit clone http://llvm.org/git/clang.git\n\n"
+"     \tIf you are using old version of clang, change 'clang-bpf-cmd-template'\n"
+"     \toption in [llvm] section of ~/.perfconfig to:\n\n"
+"     \t  \"$CLANG_EXEC $CLANG_OPTIONS $KERNEL_INC_OPTIONS \\\n"
+"     \t     -working-directory $WORKING_DIR -c $CLANG_SOURCE \\\n"
+"     \t     -emit-llvm -o - | /path/to/llc -march=bpf -filetype=obj -o -\"\n"
+"     \t(Replace /path/to/llc with path to your llc)\n\n"
+);
+}
+
+int llvm__compile_bpf(const char *path, void **p_obj_buf,
+		      size_t *p_obj_buf_sz)
+{
+	int err;
+	char clang_path[PATH_MAX];
+	const char *clang_opt = llvm_param.clang_opt;
+	const char *template = llvm_param.clang_bpf_cmd_template;
+	void *obj_buf = NULL;
+	size_t obj_buf_sz;
+
+	if (!template)
+		template = CLANG_BPF_CMD_DEFAULT_TEMPLATE;
+
+	err = search_program(llvm_param.clang_path,
+			     "clang", clang_path);
+	if (err) {
+		pr_err(
+"ERROR:\tunable to find clang.\n"
+"Hint:\tTry to install latest clang/llvm to support BPF. Check your $PATH\n"
+"     \tand 'clang-path' option in [llvm] section of ~/.perfconfig.\n");
+		version_notice();
+		return -ENOENT;
+	}
+
+	force_set_env("CLANG_EXEC", clang_path);
+	force_set_env("CLANG_OPTIONS", clang_opt);
+	force_set_env("KERNEL_INC_OPTIONS", NULL);
+	force_set_env("WORKING_DIR", ".");
+
+	/*
+	 * Since we may reset clang's working dir, path of source file
+	 * should be transferred into absolute path, except we want
+	 * stdin to be source file (testing).
+	 */
+	force_set_env("CLANG_SOURCE",
+		      (path[0] == '-') ? path :
+		      make_nonrelative_path(path));
+
+	pr_debug("llvm compiling command template: %s\n", template);
+	err = read_from_pipe(template, &obj_buf, &obj_buf_sz);
+	if (err) {
+		pr_err("ERROR:\tunable to compile %s\n", path);
+		pr_err("Hint:\tCheck error message shown above.\n");
+		version_notice();
+		pr_err("Hint:\tYou can also pre-compile it into .o\n");
+		goto errout;
+	}
+
+	if (!p_obj_buf)
+		free(obj_buf);
+	else
+		*p_obj_buf = obj_buf;
+
+	if (p_obj_buf_sz)
+		*p_obj_buf_sz = obj_buf_sz;
+	return 0;
+errout:
+	free(obj_buf);
+	if (p_obj_buf)
+		*p_obj_buf = NULL;
+	if (p_obj_buf_sz)
+		*p_obj_buf_sz = 0;
+	return err;
+}
diff --git a/tools/perf/util/llvm-utils.h b/tools/perf/util/llvm-utils.h
index 504b799..d23adbc 100644
--- a/tools/perf/util/llvm-utils.h
+++ b/tools/perf/util/llvm-utils.h
@@ -33,4 +33,7 @@ struct llvm_param {
 
 extern struct llvm_param llvm_param;
 extern int perf_llvm_config(const char *var, const char *value);
+
+extern int llvm__compile_bpf(const char *path, void **p_obj_buf,
+			     size_t *p_obj_buf_sz);
 #endif
-- 
1.8.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1174893 — [RFC PATCH v10 02/50] tracing, perf: Implement BPF programs attached to uprobes

By copying BPF related operation to uprobe processing path, this patch
allow users attach BPF programs to uprobes like what they are already
doing on kprobes.

After this patch, users are allowed to use PERF_EVENT_IOC_SET_BPF on a
uprobe perf event. Which make it possible to profile user space programs
and kernel events together using BPF.

Because of this patch, CONFIG_BPF_EVENTS should be selected by
CONFIG_UPROBE_EVENT to ensure trace_call_bpf() is compiled even if
KPROBE_EVENT is not set.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Alexei Starovoitov <ast@plumgrid.com>
---
 include/linux/ftrace_event.h | 5 +++++
 kernel/events/core.c         | 4 ++--
 kernel/trace/Kconfig         | 2 +-
 kernel/trace/trace_uprobe.c  | 5 +++++
 4 files changed, 13 insertions(+), 3 deletions(-)

diff --git a/include/linux/ftrace_event.h b/include/linux/ftrace_event.h
index ec48c92..98cb3d6 100644
--- a/include/linux/ftrace_event.h
+++ b/include/linux/ftrace_event.h
@@ -254,6 +254,7 @@ enum {
 	TRACE_EVENT_FL_USE_CALL_FILTER_BIT,
 	TRACE_EVENT_FL_TRACEPOINT_BIT,
 	TRACE_EVENT_FL_KPROBE_BIT,
+	TRACE_EVENT_FL_UPROBE_BIT,
 };
 
 /*
@@ -268,6 +269,7 @@ enum {
  *  USE_CALL_FILTER - For ftrace internal events, don't use file filter
  *  TRACEPOINT    - Event is a tracepoint
  *  KPROBE        - Event is a kprobe
+ *  UPROBE        - Event is a uprobe
  */
 enum {
 	TRACE_EVENT_FL_FILTERED		= (1 << TRACE_EVENT_FL_FILTERED_BIT),
@@ -278,8 +280,11 @@ enum {
 	TRACE_EVENT_FL_USE_CALL_FILTER	= (1 << TRACE_EVENT_FL_USE_CALL_FILTER_BIT),
 	TRACE_EVENT_FL_TRACEPOINT	= (1 << TRACE_EVENT_FL_TRACEPOINT_BIT),
 	TRACE_EVENT_FL_KPROBE		= (1 << TRACE_EVENT_FL_KPROBE_BIT),
+	TRACE_EVENT_FL_UPROBE		= (1 << TRACE_EVENT_FL_UPROBE_BIT),
 };
 
+#define TRACE_EVENT_FL_UKPROBE (TRACE_EVENT_FL_KPROBE | TRACE_EVENT_FL_UPROBE)
+
 struct ftrace_event_call {
 	struct list_head	list;
 	struct ftrace_event_class *class;
diff --git a/kernel/events/core.c b/kernel/events/core.c
index 8e13f3e..a8f75bf 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -6751,8 +6751,8 @@ static int perf_event_set_bpf_prog(struct perf_event *event, u32 prog_fd)
 	if (event->tp_event->prog)
 		return -EEXIST;
 
-	if (!(event->tp_event->flags & TRACE_EVENT_FL_KPROBE))
-		/* bpf programs can only be attached to kprobes */
+	if (!(event->tp_event->flags & TRACE_EVENT_FL_UKPROBE))
+		/* bpf programs can only be attached to u/kprobes */
 		return -EINVAL;
 
 	prog = bpf_prog_get(prog_fd);
diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
index 3b9a48a..1153c43 100644
--- a/kernel/trace/Kconfig
+++ b/kernel/trace/Kconfig
@@ -434,7 +434,7 @@ config UPROBE_EVENT
 
 config BPF_EVENTS
 	depends on BPF_SYSCALL
-	depends on KPROBE_EVENT
+	depends on KPROBE_EVENT || UPROBE_EVENT
 	bool
 	default y
 	help
diff --git a/kernel/trace/trace_uprobe.c b/kernel/trace/trace_uprobe.c
index 6dd022c..0b580bd 100644
--- a/kernel/trace/trace_uprobe.c
+++ b/kernel/trace/trace_uprobe.c
@@ -1095,11 +1095,15 @@ static void __uprobe_perf_func(struct trace_uprobe *tu,
 {
 	struct ftrace_event_call *call = &tu->tp.call;
 	struct uprobe_trace_entry_head *entry;
+	struct bpf_prog *prog = call->prog;
 	struct hlist_head *head;
 	void *data;
 	int size, esize;
 	int rctx;
 
+	if (prog && !trace_call_bpf(prog, regs))
+		return;
+
 	esize = SIZEOF_TRACE_ENTRY(is_ret_probe(tu));
 
 	size = esize + tu->tp.size + dsize;
@@ -1289,6 +1293,7 @@ static int register_uprobe_event(struct trace_uprobe *tu)
 		return -ENODEV;
 	}
 
+	call->flags = TRACE_EVENT_FL_UPROBE;
 	call->class->reg = trace_uprobe_register;
 	call->data = tu;
 	ret = trace_add_event_call(call);
-- 
1.8.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1174894 — [RFC PATCH v10 05/50] bpf tools: Open eBPF object file and do basic validation

This patch defines basic interface of libbpf. 'struct bpf_object' will
be the handler of each object file. Its internal structure is hide to
user. eBPF object files are compiled by LLVM as ELF format. In this
patch, libelf is used to open those files, read EHDR and do basic
validation according to e_type and e_machine.

All elf related staffs are grouped together and reside in efile field of
'struct bpf_object'. bpf_object__elf_finish() is introduced to clear it.

After all eBPF programs in an object file are loaded, related ELF
information is useless. Close the object file and free those memory.

zfree() and zclose() are introduced to ensure setting NULL pointers and
negative file descriptors after resources are released.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Alexei Starovoitov <ast@plumgrid.com>
---
 tools/lib/bpf/libbpf.c | 158 +++++++++++++++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf.h |   8 +++
 2 files changed, 166 insertions(+)

diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 6f0c13a..9e44608 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -11,8 +11,12 @@
 #include <stdarg.h>
 #include <string.h>
 #include <unistd.h>
+#include <fcntl.h>
+#include <errno.h>
 #include <asm/unistd.h>
 #include <linux/bpf.h>
+#include <libelf.h>
+#include <gelf.h>
 
 #include "libbpf.h"
 
@@ -52,3 +56,157 @@ void libbpf_set_print(libbpf_print_fn_t warn,
 	__pr_info = info;
 	__pr_debug = debug;
 }
+
+/* Copied from tools/perf/util/util.h */
+#ifndef zfree
+# define zfree(ptr) ({ free(*ptr); *ptr = NULL; })
+#endif
+
+#ifndef zclose
+# define zclose(fd) ({			\
+	int ___err = 0;			\
+	if ((fd) >= 0)			\
+		___err = close((fd));	\
+	fd = -1;			\
+	___err; })
+#endif
+
+#ifdef HAVE_LIBELF_MMAP_SUPPORT
+# define LIBBPF_ELF_C_READ_MMAP ELF_C_READ_MMAP
+#else
+# define LIBBPF_ELF_C_READ_MMAP ELF_C_READ
+#endif
+
+struct bpf_object {
+	/*
+	 * Information when doing elf related work. Only valid if fd
+	 * is valid.
+	 */
+	struct {
+		int fd;
+		Elf *elf;
+		GElf_Ehdr ehdr;
+	} efile;
+	char path[];
+};
+#define obj_elf_valid(o)	((o)->efile.elf)
+
+static struct bpf_object *bpf_object__new(const char *path)
+{
+	struct bpf_object *obj;
+
+	obj = calloc(1, sizeof(struct bpf_object) + strlen(path) + 1);
+	if (!obj) {
+		pr_warning("alloc memory failed for %s\n", path);
+		return NULL;
+	}
+
+	strcpy(obj->path, path);
+	obj->efile.fd = -1;
+	return obj;
+}
+
+static void bpf_object__elf_finish(struct bpf_object *obj)
+{
+	if (!obj_elf_valid(obj))
+		return;
+
+	if (obj->efile.elf) {
+		elf_end(obj->efile.elf);
+		obj->efile.elf = NULL;
+	}
+	zclose(obj->efile.fd);
+}
+
+static int bpf_object__elf_init(struct bpf_object *obj)
+{
+	int err = 0;
+	GElf_Ehdr *ep;
+
+	if (obj_elf_valid(obj)) {
+		pr_warning("elf init: internal error\n");
+		return -EEXIST;
+	}
+
+	obj->efile.fd = open(obj->path, O_RDONLY);
+	if (obj->efile.fd < 0) {
+		pr_warning("failed to open %s: %s\n", obj->path,
+				strerror(errno));
+		return -errno;
+	}
+
+	obj->efile.elf = elf_begin(obj->efile.fd,
+				 LIBBPF_ELF_C_READ_MMAP,
+				 NULL);
+	if (!obj->efile.elf) {
+		pr_warning("failed to open %s as ELF file\n",
+				obj->path);
+		err = -EINVAL;
+		goto errout;
+	}
+
+	if (!gelf_getehdr(obj->efile.elf, &obj->efile.ehdr)) {
+		pr_warning("failed to get EHDR from %s\n",
+				obj->path);
+		err = -EINVAL;
+		goto errout;
+	}
+	ep = &obj->efile.ehdr;
+
+	if ((ep->e_type != ET_REL) || (ep->e_machine != 0)) {
+		pr_warning("%s is not an eBPF object file\n",
+			obj->path);
+		err = -EINVAL;
+		goto errout;
+	}
+
+	return 0;
+errout:
+	bpf_object__elf_finish(obj);
+	return err;
+}
+
+static struct bpf_object *
+__bpf_object__open(const char *path)
+{
+	struct bpf_object *obj;
+
+	if (elf_version(EV_CURRENT) == EV_NONE) {
+		pr_warning("failed to init libelf for %s\n", path);
+		return NULL;
+	}
+
+	obj = bpf_object__new(path);
+	if (!obj)
+		return NULL;
+
+	if (bpf_object__elf_init(obj))
+		goto out;
+
+	bpf_object__elf_finish(obj);
+	return obj;
+out:
+	bpf_object__close(obj);
+	return NULL;
+}
+
+struct bpf_object *bpf_object__open(const char *path)
+{
+	/* param validation */
+	if (!path)
+		return NULL;
+
+	pr_debug("loading %s\n", path);
+
+	return __bpf_object__open(path);
+}
+
+void bpf_object__close(struct bpf_object *obj)
+{
+	if (!obj)
+		return;
+
+	bpf_object__elf_finish(obj);
+
+	free(obj);
+}
diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h
index 8d1eeba..ec3301c 100644
--- a/tools/lib/bpf/libbpf.h
+++ b/tools/lib/bpf/libbpf.h
@@ -8,6 +8,8 @@
 #ifndef __BPF_LIBBPF_H
 #define __BPF_LIBBPF_H
 
+#include <stdio.h>
+
 /*
  * In include/linux/compiler-gcc.h, __printf is defined. However
  * it should be better if libbpf.h doesn't depend on Linux header file.
@@ -20,4 +22,10 @@ void libbpf_set_print(libbpf_print_fn_t warn,
 		      libbpf_print_fn_t info,
 		      libbpf_print_fn_t debug);
 
+/* Hide internal to user */
+struct bpf_object;
+
+struct bpf_object *bpf_object__open(const char *path);
+void bpf_object__close(struct bpf_object *object);
+
 #endif
-- 
1.8.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1174895 — [RFC PATCH v10 22/50] bpf tools: Link all bpf objects onto a list

To prevent caller from creating additional structures to hold
pointers of 'struct bpf_object', this patch link all such
structures onto a list (hidden to user). bpf_object__for_each() is
introduced to allow users iterate over each objects.
bpf_object__for_each() is safe even user close the object during
iteration.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Alexei Starovoitov <ast@plumgrid.com>
---
 tools/lib/bpf/libbpf.c | 32 ++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf.h |  7 +++++++
 2 files changed, 39 insertions(+)

diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 1c210fb..540ac558 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -17,6 +17,7 @@
 #include <asm/unistd.h>
 #include <linux/kernel.h>
 #include <linux/bpf.h>
+#include <linux/list.h>
 #include <libelf.h>
 #include <gelf.h>
 
@@ -104,6 +105,8 @@ struct bpf_program {
 	bpf_program_clear_priv_t clear_priv;
 };
 
+static LIST_HEAD(bpf_objects_list);
+
 struct bpf_object {
 	char license[64];
 	u32 kern_version;
@@ -137,6 +140,12 @@ struct bpf_object {
 		} *reloc;
 		int nr_reloc;
 	} efile;
+	/*
+	 * All loaded bpf_object is linked in a list, which is
+	 * hidden to caller. bpf_objects__<func> handlers deal with
+	 * all objects.
+	 */
+	struct list_head list;
 	char path[];
 };
 #define obj_elf_valid(o)	((o)->efile.elf)
@@ -254,6 +263,9 @@ static struct bpf_object *bpf_object__new(const char *path,
 	obj->efile.obj_buf_sz = obj_buf_sz;
 
 	obj->loaded = false;
+
+	INIT_LIST_HEAD(&obj->list);
+	list_add(&obj->list, &bpf_objects_list);
 	return obj;
 }
 
@@ -933,6 +945,7 @@ void bpf_object__close(struct bpf_object *obj)
 	}
 	zfree(&obj->programs);
 
+	list_del(&obj->list);
 	free(obj);
 }
 
@@ -945,6 +958,25 @@ int bpf_object__get_prog_cnt(struct bpf_object *obj, size_t *pcnt)
 	return 0;
 }
 
+struct bpf_object *
+bpf_object__next(struct bpf_object *prev)
+{
+	struct bpf_object *next;
+
+	if (!prev)
+		next = list_first_entry(&bpf_objects_list,
+					struct bpf_object,
+					list);
+	else
+		next = list_next_entry(prev, list);
+
+	/* Empty list is noticed here so don't need checking on entry. */
+	if (&next->list == &bpf_objects_list)
+		return NULL;
+
+	return next;
+}
+
 struct bpf_program *
 bpf_program__next(struct bpf_program *prev, struct bpf_object *obj)
 {
diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h
index a20ae2e..1cd17a2 100644
--- a/tools/lib/bpf/libbpf.h
+++ b/tools/lib/bpf/libbpf.h
@@ -38,6 +38,13 @@ int bpf_object__unload(struct bpf_object *obj);
 /* Accessors of bpf_object */
 int bpf_object__get_prog_cnt(struct bpf_object *obj, size_t *pcnt);
 
+struct bpf_object *bpf_object__next(struct bpf_object *prev);
+#define bpf_object__for_each(pos, tmp)			\
+	for ((pos) = bpf_object__next(NULL),		\
+		(tmp) = bpf_object__next(pos);		\
+	     (pos) != NULL;				\
+	     (pos) = (tmp), (tmp) = bpf_object__next(tmp))
+
 /* Accessors of bpf_program. */
 struct bpf_program;
 struct bpf_program *bpf_program__next(struct bpf_program *prog,
-- 
1.8.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1174896 — [RFC PATCH v10 27/50] perf tools: Auto detecting kernel build directory

This patch detects kernel build directory using a embedded shell
script 'kbuild_detector', which does this by checking existence of
include/generated/autoconf.h.

clang working directory is changed to kbuild directory if it is found,
to help user use relative include path. Following patch will detect
kernel include directory, which contains relative include patch so this
workdir changing is needed.

Users are allowed to set 'kbuild-dir = ""' manually to disable this
checking.

Signed-off-by: Wang Nan <wangnan0@hauwei.com>
---
 tools/perf/util/llvm-utils.c | 56 +++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 55 insertions(+), 1 deletion(-)

diff --git a/tools/perf/util/llvm-utils.c b/tools/perf/util/llvm-utils.c
index 46d42c3..ea96f85 100644
--- a/tools/perf/util/llvm-utils.c
+++ b/tools/perf/util/llvm-utils.c
@@ -201,6 +201,51 @@ version_notice(void)
 );
 }
 
+static const char *kbuild_detector =
+"#!/usr/bin/env sh\n"
+"DEFAULT_KBUILD_DIR=/lib/modules/`uname -r`/build\n"
+"if test -z \"$KBUILD_DIR\"\n"
+"then\n"
+"    KBUILD_DIR=$DEFAULT_KBUILD_DIR\n"
+"fi\n"
+"if test -f $KBUILD_DIR/include/generated/autoconf.h\n"
+"then\n"
+"	echo -n $KBUILD_DIR\n"
+"	exit 0\n"
+"fi\n"
+"exit -1\n";
+
+static inline void
+get_kbuild_opts(char **kbuild_dir)
+{
+	int err;
+
+	if (!kbuild_dir)
+		return;
+
+	*kbuild_dir = NULL;
+
+	if (llvm_param.kbuild_dir && !llvm_param.kbuild_dir[0]) {
+		pr_debug("[llvm.kbuild-dir] is set to \"\" deliberately.\n");
+		pr_debug("Skip kbuild options detection.\n");
+		return;
+	}
+
+	force_set_env("KBUILD_DIR", llvm_param.kbuild_dir);
+	force_set_env("KBUILD_OPTS", llvm_param.kbuild_opts);
+	err = read_from_pipe(kbuild_detector,
+			     ((void **)kbuild_dir),
+			     NULL);
+	if (err) {
+		pr_warning(
+"WARNING:\tunable to get correct kernel building directory.\n"
+"Hint:\tSet correct kbuild directory using 'kbuild-dir' option in [llvm]\n"
+"     \tsection of ~/.perfconfig or set it to \"\" to suppress kbuild\n"
+"     \tdetection.\n\n");
+		return;
+	}
+}
+
 int llvm__compile_bpf(const char *path, void **p_obj_buf,
 		      size_t *p_obj_buf_sz)
 {
@@ -208,6 +253,7 @@ int llvm__compile_bpf(const char *path, void **p_obj_buf,
 	char clang_path[PATH_MAX];
 	const char *clang_opt = llvm_param.clang_opt;
 	const char *template = llvm_param.clang_bpf_cmd_template;
+	char *kbuild_dir = NULL;
 	void *obj_buf = NULL;
 	size_t obj_buf_sz;
 
@@ -225,10 +271,16 @@ int llvm__compile_bpf(const char *path, void **p_obj_buf,
 		return -ENOENT;
 	}
 
+	/*
+	 * This is an optional work. Even it fail we can continue our
+	 * work. Needn't to check error return.
+	 */
+	get_kbuild_opts(&kbuild_dir);
+
 	force_set_env("CLANG_EXEC", clang_path);
 	force_set_env("CLANG_OPTIONS", clang_opt);
 	force_set_env("KERNEL_INC_OPTIONS", NULL);
-	force_set_env("WORKING_DIR", ".");
+	force_set_env("WORKING_DIR", kbuild_dir ? : ".");
 
 	/*
 	 * Since we may reset clang's working dir, path of source file
@@ -249,6 +301,7 @@ int llvm__compile_bpf(const char *path, void **p_obj_buf,
 		goto errout;
 	}
 
+	free(kbuild_dir);
 	if (!p_obj_buf)
 		free(obj_buf);
 	else
@@ -258,6 +311,7 @@ int llvm__compile_bpf(const char *path, void **p_obj_buf,
 		*p_obj_buf_sz = obj_buf_sz;
 	return 0;
 errout:
+	free(kbuild_dir);
 	free(obj_buf);
 	if (p_obj_buf)
 		*p_obj_buf = NULL;
-- 
1.8.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1174897 — [RFC PATCH v10 06/50] bpf tools: Read eBPF object from buffer

To support dynamic compiling, this patch allows caller to pass a
in-memory buffer to libbpf by bpf_object__open_buffer(). libbpf
calls elf_memory() to open it as ELF object file.

Because __bpf_object__open() collects all required data and won't need
that buffer anymore, libbpf uses that buffer directly instead of clone
a new buffer. Caller of libbpf can free that buffer or use it do other
things after bpf_object__open_buffer() return.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
---
 tools/lib/bpf/libbpf.c | 62 ++++++++++++++++++++++++++++++++++++++++----------
 tools/lib/bpf/libbpf.h |  2 ++
 2 files changed, 52 insertions(+), 12 deletions(-)

diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 9e44608..36dfbc1 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -84,6 +84,8 @@ struct bpf_object {
 	 */
 	struct {
 		int fd;
+		void *obj_buf;
+		size_t obj_buf_sz;
 		Elf *elf;
 		GElf_Ehdr ehdr;
 	} efile;
@@ -91,7 +93,9 @@ struct bpf_object {
 };
 #define obj_elf_valid(o)	((o)->efile.elf)
 
-static struct bpf_object *bpf_object__new(const char *path)
+static struct bpf_object *bpf_object__new(const char *path,
+					  void *obj_buf,
+					  size_t obj_buf_sz)
 {
 	struct bpf_object *obj;
 
@@ -103,6 +107,16 @@ static struct bpf_object *bpf_object__new(const char *path)
 
 	strcpy(obj->path, path);
 	obj->efile.fd = -1;
+
+	/*
+	 * Caller of this function should also calls
+	 * bpf_object__elf_finish() after data collection to return
+	 * obj_buf to user. If not, we should duplicate the buffer to
+	 * avoid user freeing them before elf finish.
+	 */
+	obj->efile.obj_buf = obj_buf;
+	obj->efile.obj_buf_sz = obj_buf_sz;
+
 	return obj;
 }
 
@@ -116,6 +130,8 @@ static void bpf_object__elf_finish(struct bpf_object *obj)
 		obj->efile.elf = NULL;
 	}
 	zclose(obj->efile.fd);
+	obj->efile.obj_buf = NULL;
+	obj->efile.obj_buf_sz = 0;
 }
 
 static int bpf_object__elf_init(struct bpf_object *obj)
@@ -128,16 +144,26 @@ static int bpf_object__elf_init(struct bpf_object *obj)
 		return -EEXIST;
 	}
 
-	obj->efile.fd = open(obj->path, O_RDONLY);
-	if (obj->efile.fd < 0) {
-		pr_warning("failed to open %s: %s\n", obj->path,
-				strerror(errno));
-		return -errno;
+	if (obj->efile.obj_buf_sz > 0) {
+		/*
+		 * obj_buf should have been validated by
+		 * bpf_object__open_buffer().
+		 */
+		obj->efile.elf = elf_memory(obj->efile.obj_buf,
+					    obj->efile.obj_buf_sz);
+	} else {
+		obj->efile.fd = open(obj->path, O_RDONLY);
+		if (obj->efile.fd < 0) {
+			pr_warning("failed to open %s: %s\n", obj->path,
+					strerror(errno));
+			return -errno;
+		}
+
+		obj->efile.elf = elf_begin(obj->efile.fd,
+				LIBBPF_ELF_C_READ_MMAP,
+				NULL);
 	}
 
-	obj->efile.elf = elf_begin(obj->efile.fd,
-				 LIBBPF_ELF_C_READ_MMAP,
-				 NULL);
 	if (!obj->efile.elf) {
 		pr_warning("failed to open %s as ELF file\n",
 				obj->path);
@@ -167,7 +193,7 @@ errout:
 }
 
 static struct bpf_object *
-__bpf_object__open(const char *path)
+__bpf_object__open(const char *path, void *obj_buf, size_t obj_buf_sz)
 {
 	struct bpf_object *obj;
 
@@ -176,7 +202,7 @@ __bpf_object__open(const char *path)
 		return NULL;
 	}
 
-	obj = bpf_object__new(path);
+	obj = bpf_object__new(path, obj_buf, obj_buf_sz);
 	if (!obj)
 		return NULL;
 
@@ -198,7 +224,19 @@ struct bpf_object *bpf_object__open(const char *path)
 
 	pr_debug("loading %s\n", path);
 
-	return __bpf_object__open(path);
+	return __bpf_object__open(path, NULL, 0);
+}
+
+struct bpf_object *bpf_object__open_buffer(void *obj_buf,
+					   size_t obj_buf_sz)
+{
+	/* param validation */
+	if (!obj_buf || obj_buf_sz <= 0)
+		return NULL;
+
+	pr_debug("loading object from buffer\n");
+
+	return __bpf_object__open("[buffer]", obj_buf, obj_buf_sz);
 }
 
 void bpf_object__close(struct bpf_object *obj)
diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h
index ec3301c..dc966dd 100644
--- a/tools/lib/bpf/libbpf.h
+++ b/tools/lib/bpf/libbpf.h
@@ -26,6 +26,8 @@ void libbpf_set_print(libbpf_print_fn_t warn,
 struct bpf_object;
 
 struct bpf_object *bpf_object__open(const char *path);
+struct bpf_object *bpf_object__open_buffer(void *obj_buf,
+					   size_t obj_buf_sz);
 void bpf_object__close(struct bpf_object *object);
 
 #endif
-- 
1.8.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1174898 — [RFC PATCH v10 13/50] bpf tools: Collect relocation sections from SHT_REL sections

This patch collects relocation sections into 'struct object'.
Such sections are used for connecting maps to bpf programs.
'reloc' field in 'struct bpf_object' is introduced for storing
such information.

This patch simply store the data into 'reloc' field. Following
patch will parse them to know the exact instructions which are
needed to be relocated.

Note that the collected data will be invalid after ELF object file
is closed.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Alexei Starovoitov <ast@plumgrid.com>
---
 tools/lib/bpf/libbpf.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 9c01b55f..f3ace51 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -110,6 +110,11 @@ struct bpf_object {
 		Elf *elf;
 		GElf_Ehdr ehdr;
 		Elf_Data *symbols;
+		struct {
+			GElf_Shdr shdr;
+			Elf_Data *data;
+		} *reloc;
+		int nr_reloc;
 	} efile;
 	char path[];
 };
@@ -220,6 +225,9 @@ static void bpf_object__elf_finish(struct bpf_object *obj)
 		obj->efile.elf = NULL;
 	}
 	obj->efile.symbols = NULL;
+
+	zfree(&obj->efile.reloc);
+	obj->efile.nr_reloc = 0;
 	zclose(obj->efile.fd);
 	obj->efile.obj_buf = NULL;
 	obj->efile.obj_buf_sz = 0;
@@ -440,6 +448,24 @@ static int bpf_object__elf_collect(struct bpf_object *obj)
 			} else
 				pr_debug("found program %s\n",
 					 prog->section_name);
+		} else if (sh.sh_type == SHT_REL) {
+			void *reloc = obj->efile.reloc;
+			int nr_reloc = obj->efile.nr_reloc + 1;
+
+			reloc = realloc(reloc,
+					sizeof(*obj->efile.reloc) * nr_reloc);
+			if (!reloc) {
+				pr_warning("realloc failed\n");
+				err = -ENOMEM;
+			} else {
+				int n = nr_reloc - 1;
+
+				obj->efile.reloc = reloc;
+				obj->efile.nr_reloc = nr_reloc;
+
+				obj->efile.reloc[n].shdr = sh;
+				obj->efile.reloc[n].data = data;
+			}
 		}
 		if (err)
 			goto out;
-- 
1.8.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1174899 — [RFC PATCH v10 01/50] bpf: Use correct #ifdef controller for trace_call_bpf()

Commit e1abf2cc8d5d80b41c4419368ec743ccadbb131e ("bpf: Fix the build on
BPF_SYSCALL=y && !CONFIG_TRACING kernels, make it more configurable")
updated the building condition of bpf_trace.o from CONFIG_BPF_SYSCALL
to CONFIG_BPF_EVENTS, but the corresponding #ifdef controller in
ftrace_event.h for trace_call_bpf() is not changed. Which, in theory,
is incorrect.

With current Kconfigs, we can create a .config with CONFIG_BPF_SYSCALL=y
and CONFIG_BPF_EVENTS=n by unselecting CONFIG_KPROBE_EVENT and
selecting CONFIG_BPF_SYSCALL. With these options, trace_call_bpf() will
be defined as an extern function, but if anyone calls it a symbol missing
error will be trigger since bpf_trace.o is not build.

This patch changes the #ifdef controller for trace_call_bpf() from
CONFIG_BPF_SYSCALL to CONFIG_BPF_EVENTS. I'll show its correctness:

Before this patch:

   BPF_SYSCALL   BPF_EVENTS   trace_call_bpf   bpf_trace.o
   y             y           normal           compiled
   n             n           inline           not compiled
   y             n           normal           not compiled (incorrect)
   n             y          impossible (BPF_EVENTS depends on BPF_SYSCALL)

After this patch:

   BPF_SYSCALL   BPF_EVENTS   trace_call_bpf   bpf_trace.o
   y             y           normal           compiled
   n             n           inline           not compiled
   y             n           inline           not compiled (fixed)
   n             y          impossible (BPF_EVENTS depends on BPF_SYSCALL)

So this patch doesn't break thing. QED.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
---
 include/linux/ftrace_event.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/linux/ftrace_event.h b/include/linux/ftrace_event.h
index f9ecf63..ec48c92 100644
--- a/include/linux/ftrace_event.h
+++ b/include/linux/ftrace_event.h
@@ -553,7 +553,7 @@ event_trigger_unlock_commit_regs(struct ftrace_event_file *file,
 		event_triggers_post_call(file, tt);
 }
 
-#ifdef CONFIG_BPF_SYSCALL
+#ifdef CONFIG_BPF_EVENTS
 unsigned int trace_call_bpf(struct bpf_prog *prog, void *ctx);
 #else
 static inline unsigned int trace_call_bpf(struct bpf_prog *prog, void *ctx)
-- 
1.8.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1174900 — [RFC PATCH v10 16/50] bpf tools: Create eBPF maps defined in an object file

This patch creates maps based on 'map' section in object file using
bpf_create_map(), and stores the fds into an array in
'struct bpf_object'.

Previous patches parse ELF object file and collecte required data, but
doesn't play with kernel. They belong to 'opening' phase. This patch is
the first patch in 'loading' phase. 'loaded' field is introduced to
'struct bpf_object' to avoid loading one object twice, because loading
phase clears resources collected during opening which become useless
after loading. In this patch, maps_buf is cleared.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Alexei Starovoitov <ast@plumgrid.com>
---
 tools/lib/bpf/libbpf.c | 102 +++++++++++++++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf.h |   4 ++
 2 files changed, 106 insertions(+)

diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index f16234e..dee6f41 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -21,6 +21,7 @@
 #include <gelf.h>
 
 #include "libbpf.h"
+#include "bpf.h"
 
 #define __printf(a, b)	__attribute__((format(printf, a, b)))
 
@@ -105,6 +106,13 @@ struct bpf_object {
 
 	struct bpf_program *programs;
 	size_t nr_programs;
+	int *map_fds;
+	/*
+	 * This field is required because maps_buf will be freed and
+	 * maps_buf_sz will be set to 0 after loaded.
+	 */
+	size_t nr_map_fds;
+	bool loaded;
 
 	/*
 	 * Information when doing elf related work. Only valid if fd
@@ -222,6 +230,7 @@ static struct bpf_object *bpf_object__new(const char *path,
 	obj->efile.obj_buf = obj_buf;
 	obj->efile.obj_buf_sz = obj_buf_sz;
 
+	obj->loaded = false;
 	return obj;
 }
 
@@ -558,6 +567,62 @@ bpf_program__collect_reloc(struct bpf_program *prog,
 	return 0;
 }
 
+static int
+bpf_object__create_maps(struct bpf_object *obj)
+{
+	unsigned int i;
+	size_t nr_maps;
+	int *pfd;
+
+	nr_maps = obj->maps_buf_sz / sizeof(struct bpf_map_def);
+	if (!obj->maps_buf || !nr_maps) {
+		pr_debug("don't need create maps for %s\n",
+			 obj->path);
+		return 0;
+	}
+
+	obj->map_fds = malloc(sizeof(int) * nr_maps);
+	if (!obj->map_fds) {
+		pr_warning("realloc perf_bpf_map_fds failed\n");
+		return -ENOMEM;
+	}
+	obj->nr_map_fds = nr_maps;
+
+	/* fill all fd with -1 */
+	memset(obj->map_fds, 0xff, sizeof(int) * nr_maps);
+
+	pfd = obj->map_fds;
+	for (i = 0; i < nr_maps; i++) {
+		struct bpf_map_def def;
+
+		def = *(struct bpf_map_def *)(obj->maps_buf +
+				i * sizeof(struct bpf_map_def));
+
+		*pfd = bpf_create_map(def.type,
+				      def.key_size,
+				      def.value_size,
+				      def.max_entries);
+		if (*pfd < 0) {
+			size_t j;
+			int err = *pfd;
+
+			pr_warning("failed to create map: %s\n",
+				   strerror(errno));
+			for (j = 0; j < i; j++)
+				zclose(obj->map_fds[j]);
+			obj->nr_map_fds = 0;
+			zfree(&obj->map_fds);
+			return err;
+		}
+		pr_debug("create map: fd=%d\n", *pfd);
+		pfd++;
+	}
+
+	zfree(&obj->maps_buf);
+	obj->maps_buf_sz = 0;
+	return 0;
+}
+
 static int bpf_object__collect_reloc(struct bpf_object *obj)
 {
 	int i, err;
@@ -661,6 +726,42 @@ struct bpf_object *bpf_object__open_buffer(void *obj_buf,
 	return __bpf_object__open("[buffer]", obj_buf, obj_buf_sz);
 }
 
+int bpf_object__unload(struct bpf_object *obj)
+{
+	size_t i;
+
+	if (!obj)
+		return -EINVAL;
+
+	for (i = 0; i < obj->nr_map_fds; i++)
+		zclose(obj->map_fds[i]);
+	zfree(&obj->map_fds);
+	obj->nr_map_fds = 0;
+
+	return 0;
+}
+
+int bpf_object__load(struct bpf_object *obj)
+{
+	if (!obj)
+		return -EINVAL;
+
+	if (obj->loaded) {
+		pr_warning("object should not be loaded twice\n");
+		return -EINVAL;
+	}
+
+	obj->loaded = true;
+	if (bpf_object__create_maps(obj))
+		goto out;
+
+	return 0;
+out:
+	bpf_object__unload(obj);
+	pr_warning("failed to load object '%s'\n", obj->path);
+	return -EINVAL;
+}
+
 void bpf_object__close(struct bpf_object *obj)
 {
 	size_t i;
@@ -669,6 +770,7 @@ void bpf_object__close(struct bpf_object *obj)
 		return;
 
 	bpf_object__elf_finish(obj);
+	bpf_object__unload(obj);
 
 	zfree(&obj->maps_buf);
 
diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h
index 6e75acd..3e69600 100644
--- a/tools/lib/bpf/libbpf.h
+++ b/tools/lib/bpf/libbpf.h
@@ -30,6 +30,10 @@ struct bpf_object *bpf_object__open_buffer(void *obj_buf,
 					   size_t obj_buf_sz);
 void bpf_object__close(struct bpf_object *object);
 
+/* Load/unload object into/from kernel */
+int bpf_object__load(struct bpf_object *obj);
+int bpf_object__unload(struct bpf_object *obj);
+
 /*
  * We don't need __attribute__((packed)) now since it is
  * unnecessary for 'bpf_map_def' because they are all aligned.
-- 
1.8.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1174901 — [RFC PATCH v10 17/50] bpf tools: Relocate eBPF programs

If an eBPF program accesses a map, LLVM generates a load instruction
which loads an absolute address into a register, like this:

  ld_64   r1, <MCOperand Expr:(mymap)>
  ...
  call    2

That ld_64 instruction will be recorded in relocation section.
To enable the usage of that map, relocation must be done by replacing
the immediate value by real map file descriptor so it can be found by
eBPF map functions.

This patch to the relocation work based on information collected by
patch 'bpf tools: Collect relocation sections from SHT_REL sections'.
For each instruction which needs relocation, it inject corresponding
file descriptor to imm field. As a part of protocol, src_reg is set to
BPF_PSEUDO_MAP_FD to notify kernel this is a map loading instruction.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Alexei Starovoitov <ast@plumgrid.com>
---
 tools/lib/bpf/libbpf.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 52 insertions(+)

diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index dee6f41..9d42774 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -623,6 +623,56 @@ bpf_object__create_maps(struct bpf_object *obj)
 	return 0;
 }
 
+static int
+bpf_program__relocate(struct bpf_program *prog, int *map_fds)
+{
+	int i;
+
+	if (!prog || !prog->reloc_desc)
+		return 0;
+
+	for (i = 0; i < prog->nr_reloc; i++) {
+		int insn_idx, map_idx;
+		struct bpf_insn *insns = prog->insns;
+
+		insn_idx = prog->reloc_desc[i].insn_idx;
+		map_idx = prog->reloc_desc[i].map_idx;
+
+		if (insn_idx >= (int)prog->insns_cnt) {
+			pr_warning("relocation out of range: '%s'\n",
+				   prog->section_name);
+			return -ERANGE;
+		}
+		insns[insn_idx].src_reg = BPF_PSEUDO_MAP_FD;
+		insns[insn_idx].imm = map_fds[map_idx];
+	}
+
+	zfree(&prog->reloc_desc);
+	prog->nr_reloc = 0;
+	return 0;
+}
+
+
+static int
+bpf_object__relocate(struct bpf_object *obj)
+{
+	struct bpf_program *prog;
+	size_t i;
+	int err;
+
+	for (i = 0; i < obj->nr_programs; i++) {
+		prog = &obj->programs[i];
+
+		err = bpf_program__relocate(prog, obj->map_fds);
+		if (err) {
+			pr_warning("failed to relocate '%s'\n",
+				   prog->section_name);
+			return err;
+		}
+	}
+	return 0;
+}
+
 static int bpf_object__collect_reloc(struct bpf_object *obj)
 {
 	int i, err;
@@ -754,6 +804,8 @@ int bpf_object__load(struct bpf_object *obj)
 	obj->loaded = true;
 	if (bpf_object__create_maps(obj))
 		goto out;
+	if (bpf_object__relocate(obj))
+		goto out;
 
 	return 0;
 out:
-- 
1.8.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1179768 — Re: [RFC PATCH v10 23/50] perf tools: Make perf depend on libbpf

Em Wed, Jul 08, 2015 at 07:45:34PM +0800, Wangnan (F) escreveu:
> On 2015/7/8 4:16, Arnaldo Carvalho de Melo wrote:
> >Em Tue, Jul 07, 2015 at 04:54:52PM -0300, Arnaldo Carvalho de Melo escreveu:
> >>Em Wed, Jul 01, 2015 at 02:14:11AM +0000, Wang Nan escreveu:
> >>>Error messages are also updated to notify users about the disable of
> >>>BPF support of 'perf record' if libelf is missed or BPF API check
> >>>failed.

> >>Much better!

> >But... I was all happy about this being linked with perf, went straight
> >ahead to try to use it! No, its not possible, I have to go thru a series
> >of other patches first... anticlimactic :-(

> >So, please move this to just before we can use it, wiring it up should
> >mean, hey, try this "hello, world" eBPF program right now!

> It is not an easy work, since there is a bulk of code in
> tools/perf/utils/bpf-loader.c depend on HAVE_LIBBPF_SUPPORT and
> CONFIG_LIBBPF. If put this patch the final one, we will make hundreds
> of lines of code avaiable by one patch.  It is not good.

Understood, makes sense, you are building the perf support step by step,
so it is not possible to test it in the first, only compile test the
initial wiring.
 
> I have an idea that, put this patch after the llvm tester:
 
> $ git log --oneline
> d011a28 perf tools: Make perf depend on libbpf
> 57ad12f perf tests: Add LLVM test for eBPF on-the-fly compiling
> 8c7e20b perf tools: Auto detecting kernel include options
> 442675f perf tools: Auto detecting kernel build directory
> dcd9304 perf tools: Call clang to compile C source to object code
> 864e2fb perf tools: Introduce llvm config options
> 8558c38 bpf tools: Link all bpf objects onto a list
> ...
 
> Then before this patch, you can test llvm on-the-fly compiling without
> parsing the
> result .o:
 
> $ perf test 38
> 38: Test LLVM searching and compiling                        : (skip bpf
> parsing) Ok
 
> After this patch, libbpf should be compiled so basic libbpf parsing can be
> tested:
> 
> $ perf test 38
> 38: Test LLVM searching and compiling                        : Ok

That looks enough for me, 'perf test' should exercise the code, making
sure that what was done up to that point got some testing.

We need to try to avoid adding too much code that doesn't get exercised
as soon as possible, otherwise we lose a lot of bisectability, i.e. when
trying to find a problem using 'git bisect' we will hit the first place
when all that code gets used and will be left with having to look at all
the code up to that point instead of landing in a more likely culprit.

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1179775 — Re: [RFC PATCH v10 23/50] perf tools: Make perf depend on libbpf

Em Tue, Jul 07, 2015 at 07:03:50PM -0700, Alexei Starovoitov escreveu:
> On 7/7/15 1:16 PM, Arnaldo Carvalho de Melo wrote:
> >So, please move this to just before we can use it, wiring it up should
> >mean, hey, try this "hello, world" eBPF program right now!
> 
> btw, since bpf is now stable llvm backend, one can just get the latest
> clang/llvm 3.7 from pre-built llvm packages for debian/ubuntu:
> http://llvm.org/apt/
> and bpf backend will be there by default.
> No need to build clang/llvm from sources.

Are there any for RHEL7 or Fedora21/22? 8-)

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web