Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1383616 > unrolled thread

[RFC PATCH 00/13] perf tools: Support uBPF script

Back to article view | Back to linux.kernel

Contents

  [RFC PATCH 00/13] perf tools: Support uBPF script Wang Nan <wangnan0@huawei.com> - 2016-04-20 20:10 +0200
    Re: [RFC PATCH 00/13] perf tools: Support uBPF script Alexei Starovoitov <alexei.starovoitov@gmail.com> - 2016-04-21 00:10 +0200
      Re: [RFC PATCH 00/13] perf tools: Support uBPF script "Wangnan (F)" <wangnan0@huawei.com> - 2016-04-21 10:20 +0200

#1383616 — [RFC PATCH 00/13] perf tools: Support uBPF script

This patch set allows to perf invoke some user space BPF scripts on some
point. uBPF scripts and kernel BPF scripts reside in one BPF object.
They communicate with each other with BPF maps. uBPF scripts can invoke
helper functions provided by perf.

At least following new features can be achieved based on uBPF support:

 1) Report statistical result:
    Like DTrace, perf print statistical report before quit. No need to
    extract data using 'perf report'. Statistical method is controled by
    user.

 2) Control perf's behavior:
    Dynamically adjust period of different events. Policy is defined by
    user.

uBPF library is required before compile. It can be found from github:

 https://github.com/iovisor/ubpf.git

 Following is an example:

 Using BPF script attached at the bottom of this commit message, one
 can print histogram of write size before perf exit like this:

 # ~/perf record  -a -e ./test_ubpf.c &
 [1] 16800
 # dd if=/dev/zero of=/dev/null bs=512 count=5000
 5000+0 records in
 5000+0 records out
 2560000 bytes (2.6 MB) copied, 0.00552838 s, 463 MB/s
 # dd if=/dev/zero of=/dev/null bs=2048 count=5000
 5000+0 records in
 5000+0 records out
 10240000 bytes (10 MB) copied, 0.0188971 s, 542 MB/s
 # fg
 ^C         <--- *Press Ctrl-c*
 2^^0: 47
 2^^1: 13
 2^^2: 4
 2^^3: 130
 2^^4: 11
 2^^5: 1051
 2^^6: 486
 2^^7: 4863
 2^^8: 0
 2^^9: 5003
 2^^10: 4
 2^^11: 5003
 2^^12: 1
 2^^13: 0
 2^^14: 0
 2^^15: 0
 2^^16: 0
 2^^17: 0
 2^^18: 0
 2^^19: 0
 2^^20: 0
 [ perf record: Woken up 1 times to write data ]
 [ perf record: Captured and wrote 0.788 MB perf.data ]

 Here is test_ubpf.c.

 /************ BEGIN ***************/
 #include <uapi/linux/bpf.h>
 #define SEC(NAME) __attribute__((section(NAME), used))
 struct bpf_map_def {
    unsigned int type;
    unsigned int key_size;
    unsigned int value_size;
    unsigned int max_entries;
 };
 
 #define BPF_ANY 0
 
 static void *(*map_lookup_elem)(struct bpf_map_def *, void *) =
                 (void *)BPF_FUNC_map_lookup_elem;
 
 static inline unsigned int log2(unsigned int v)
 {
     unsigned int r;
     unsigned int shift;
     
     r = (v > 0xFFFF) << 4; v >>= r;
     shift = (v > 0xFF) << 3; v >>= shift; r |= shift;
     shift = (v > 0xF) << 2; v >>= shift; r |= shift;
     shift = (v > 0x3) << 1; v >>= shift; r |= shift;
     r |= (v >> 1);
     return r;
 }
 
 static inline unsigned int log2l(unsigned long v)
 {
     unsigned int hi = v >> 32;
     if (hi)
         return log2(hi) + 32;
     else
         return log2(v);
 }
 
 struct bpf_map_def SEC("maps") my_hist_map = {
     .type = BPF_MAP_TYPE_ARRAY,
     .key_size = sizeof(int),
     .value_size = sizeof(long),
     .max_entries = 21,
 };
 
 SEC("sys_write=sys_write count")
 int sys_write(void *ctx, int err, long write_size)
 {
     long *value;
     int key = 0;
 
     if (err)
         return 0;
 
     key = log2l(write_size);
     if (key > 20)
         key = 20;
     value = map_lookup_elem(&my_hist_map, &key);
     if (!value)
         return 0;
     __sync_fetch_and_add(value, 1);
     return 0;
 }
 char _license[] SEC("license") = "GPL";
 u32 _version SEC("version") = LINUX_VERSION_CODE;
 
 /* Following ugly magic numbers can be find from tools/perf/util/ubpf-helpers-list.h */
 static int (*ubpf_memcmp)(void *s1, void *s2, unsigned int n) = (void *)1;
 static void (*ubpf_memcpy)(void *d, void *s, unsigned int size) = (void *)2;
 static int (*ubpf_strcmp)(void *s1, void *s2) = (void *)3;
 static int (*ubpf_printf)(char *fmt, ...) = (void *)4;
 static int (*ubpf_map_lookup_elem)(void *map_desc, void *key, void *value) = (void *)5;
 static int (*ubpf_map_update_elem)(void *map_desc, void *key, void *value, unsigned long long flags) = (void *)6;
 static int (*ubpf_map_get_next_key)(void *map_desc, void *key, void *value) = (void *)7;
 
 SEC("UBPF;perf_record_end")
 int perf_record_end(int samples)
 {
     int i, key;
     long value;
     char fmt[] = "2^^%d: %d\n";
 
     for (i = 0; i < 21; i++) {
         ubpf_map_lookup_elem(&my_hist_map, &i, &value);
         ubpf_printf(fmt, i, value);
     }
     return 0;
 }
 /************** END ***************/

Cc: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Li Zefan <lizefan@huawei.com>

Wang Nan (13):
  bpf tools: Add map related BPF helper
  tools: Add ubpf feature test
  bpf tools: Add ubpf include and makefile options
  bpf tools: Replace fd array to union array
  bpf tools: Save engine type in bpf_program
  bpf tools: Introduce ubpf_vm to program instance union
  bpf tools: Load ubpf program
  bpf tools: Add API for fetching ubpf_vm
  bpf tools: Register extern functions for ubpf programs
  perf tools: Register basic UBPF helpers
  perf bpf: Accept ubpf programs
  perf record: Add UBPF hooks at beginning and end of perf record
  perf tests: Add UBPF test case

 tools/build/Makefile.feature            |   6 +-
 tools/build/feature/Makefile            |   6 +-
 tools/build/feature/test-ubpf.c         |  11 ++
 tools/lib/bpf/Makefile                  |  16 +-
 tools/lib/bpf/bpf.c                     |  24 +++
 tools/lib/bpf/bpf.h                     |   2 +
 tools/lib/bpf/libbpf.c                  | 281 ++++++++++++++++++++++++++++----
 tools/lib/bpf/libbpf.h                  |  21 ++-
 tools/perf/Makefile.perf                |   2 +
 tools/perf/builtin-record.c             |   4 +
 tools/perf/config/Makefile              |   7 +
 tools/perf/perf.c                       |   3 +
 tools/perf/tests/Build                  |   8 +
 tools/perf/tests/bpf-script-test-ubpf.c |  68 ++++++++
 tools/perf/tests/bpf.c                  |  77 ++++++++-
 tools/perf/tests/llvm.c                 |   4 +
 tools/perf/tests/llvm.h                 |   2 +
 tools/perf/util/Build                   |   1 +
 tools/perf/util/bpf-loader.c            |  22 +++
 tools/perf/util/ubpf-helpers-list.h     |   7 +
 tools/perf/util/ubpf-helpers.c          |  66 ++++++++
 tools/perf/util/ubpf-helpers.h          |  21 +++
 tools/perf/util/ubpf-hooks-list.h       |  10 ++
 tools/perf/util/ubpf-hooks.c            |  81 +++++++++
 tools/perf/util/ubpf-hooks.h            |  35 ++++
 25 files changed, 745 insertions(+), 40 deletions(-)
 create mode 100644 tools/build/feature/test-ubpf.c
 create mode 100644 tools/perf/tests/bpf-script-test-ubpf.c
 create mode 100644 tools/perf/util/ubpf-helpers-list.h
 create mode 100644 tools/perf/util/ubpf-helpers.c
 create mode 100644 tools/perf/util/ubpf-helpers.h
 create mode 100644 tools/perf/util/ubpf-hooks-list.h
 create mode 100644 tools/perf/util/ubpf-hooks.c
 create mode 100644 tools/perf/util/ubpf-hooks.h

-- 
1.8.3.4

[toc] | [next] | [standalone]

#1383768

On Wed, Apr 20, 2016 at 06:01:40PM +0000, Wang Nan wrote:
> This patch set allows to perf invoke some user space BPF scripts on some
> point. uBPF scripts and kernel BPF scripts reside in one BPF object.
> They communicate with each other with BPF maps. uBPF scripts can invoke
> helper functions provided by perf.
> 
> At least following new features can be achieved based on uBPF support:
> 
>  1) Report statistical result:
>     Like DTrace, perf print statistical report before quit. No need to
>     extract data using 'perf report'. Statistical method is controled by
>     user.
> 
>  2) Control perf's behavior:
>     Dynamically adjust period of different events. Policy is defined by
>     user.
> 
> uBPF library is required before compile. It can be found from github:
> 
>  https://github.com/iovisor/ubpf.git
> 
>  Following is an example:
> 
>  Using BPF script attached at the bottom of this commit message, one
>  can print histogram of write size before perf exit like this:
> 
>  # ~/perf record  -a -e ./test_ubpf.c &
>  [1] 16800
>  # dd if=/dev/zero of=/dev/null bs=512 count=5000
>  5000+0 records in
>  5000+0 records out
>  2560000 bytes (2.6 MB) copied, 0.00552838 s, 463 MB/s
>  # dd if=/dev/zero of=/dev/null bs=2048 count=5000
>  5000+0 records in
>  5000+0 records out
>  10240000 bytes (10 MB) copied, 0.0188971 s, 542 MB/s
>  # fg
>  ^C         <--- *Press Ctrl-c*
>  2^^0: 47
>  2^^1: 13
>  2^^2: 4
>  2^^3: 130
>  2^^4: 11
>  2^^5: 1051
>  2^^6: 486
>  2^^7: 4863
>  2^^8: 0
>  2^^9: 5003
>  2^^10: 4
>  2^^11: 5003
>  2^^12: 1
>  2^^13: 0
>  2^^14: 0
>  2^^15: 0
>  2^^16: 0
>  2^^17: 0
>  2^^18: 0
>  2^^19: 0
>  2^^20: 0
>  [ perf record: Woken up 1 times to write data ]
>  [ perf record: Captured and wrote 0.788 MB perf.data ]
> 
>  Here is test_ubpf.c.
> 
>  /************ BEGIN ***************/
>  #include <uapi/linux/bpf.h>
>  #define SEC(NAME) __attribute__((section(NAME), used))
>  struct bpf_map_def {
>     unsigned int type;
>     unsigned int key_size;
>     unsigned int value_size;
>     unsigned int max_entries;
>  };
>  
>  #define BPF_ANY 0
>  
>  static void *(*map_lookup_elem)(struct bpf_map_def *, void *) =
>                  (void *)BPF_FUNC_map_lookup_elem;
>  
>  static inline unsigned int log2(unsigned int v)
>  {
>      unsigned int r;
>      unsigned int shift;
>      
>      r = (v > 0xFFFF) << 4; v >>= r;
>      shift = (v > 0xFF) << 3; v >>= shift; r |= shift;
>      shift = (v > 0xF) << 2; v >>= shift; r |= shift;
>      shift = (v > 0x3) << 1; v >>= shift; r |= shift;
>      r |= (v >> 1);
>      return r;
>  }
>  
>  static inline unsigned int log2l(unsigned long v)
>  {
>      unsigned int hi = v >> 32;
>      if (hi)
>          return log2(hi) + 32;
>      else
>          return log2(v);
>  }
>  
>  struct bpf_map_def SEC("maps") my_hist_map = {
>      .type = BPF_MAP_TYPE_ARRAY,
>      .key_size = sizeof(int),
>      .value_size = sizeof(long),
>      .max_entries = 21,
>  };
>  
>  SEC("sys_write=sys_write count")
>  int sys_write(void *ctx, int err, long write_size)
>  {
>      long *value;
>      int key = 0;
>  
>      if (err)
>          return 0;
>  
>      key = log2l(write_size);
>      if (key > 20)
>          key = 20;
>      value = map_lookup_elem(&my_hist_map, &key);
>      if (!value)
>          return 0;
>      __sync_fetch_and_add(value, 1);
>      return 0;
>  }
>  char _license[] SEC("license") = "GPL";
>  u32 _version SEC("version") = LINUX_VERSION_CODE;
>  
>  /* Following ugly magic numbers can be find from tools/perf/util/ubpf-helpers-list.h */
>  static int (*ubpf_memcmp)(void *s1, void *s2, unsigned int n) = (void *)1;
>  static void (*ubpf_memcpy)(void *d, void *s, unsigned int size) = (void *)2;
>  static int (*ubpf_strcmp)(void *s1, void *s2) = (void *)3;
>  static int (*ubpf_printf)(char *fmt, ...) = (void *)4;
>  static int (*ubpf_map_lookup_elem)(void *map_desc, void *key, void *value) = (void *)5;
>  static int (*ubpf_map_update_elem)(void *map_desc, void *key, void *value, unsigned long long flags) = (void *)6;
>  static int (*ubpf_map_get_next_key)(void *map_desc, void *key, void *value) = (void *)7;
>  
>  SEC("UBPF;perf_record_end")
>  int perf_record_end(int samples)
>  {
>      int i, key;
>      long value;
>      char fmt[] = "2^^%d: %d\n";
>  
>      for (i = 0; i < 21; i++) {
>          ubpf_map_lookup_elem(&my_hist_map, &i, &value);
>          ubpf_printf(fmt, i, value);
>      }
>      return 0;
>  }

Interesting!
If bpf is used for both kernel and user side programs, we can allow
almost arbitrary C code for the user side.
There is no need to be limited to a fixed set of helpers.
There is no verifier in user space either.
Just call 'printf("string")' directly.
Wouldn't even need to change interpreter.
Also ubpf was written from scratch with apache2, while perf is gpl,
so you can just link kernel/bpf/core.o directly instead of using external
libraries.
I really meant link .o file compiled for kernel.
Advertize dummy kfree/kmalloc and it will link fine, since perf
will only be calling __bpf_prog_run() which is 99% indepdendent from kernel.
I used to do exactly that long ago while performance tunning the interpreter.
Another option is to fork the interpreter for perf, but I don't like it at all.
Compiling the same bpf/core.c once for kernel and once for perf is another option,
but imo linking core.o is easier.

In general this set and overall bpf in user space makes sense only
if we allow much more flexible C code for user space.
If it's limited to ubpf_* helpers, that will quickly become suboptimal.
Another alternative is to use luajit for user space scripting like
we do in bcc. That gives full flexibility with good performance.
If we can do 'restricted C into bpf' for kernel and 'full C into bpf'
for user space that would be a great model. Note llvm doesn't care
how C looks like. You can call any function in C and use loops.

[toc] | [prev] | [next] | [standalone]

#1383924

On 2016/4/21 6:06, Alexei Starovoitov wrote:
> On Wed, Apr 20, 2016 at 06:01:40PM +0000, Wang Nan wrote:
>> This patch set allows to perf invoke some user space BPF scripts on some
>> point. uBPF scripts and kernel BPF scripts reside in one BPF object.
>> They communicate with each other with BPF maps. uBPF scripts can invoke
>> helper functions provided by perf.
>>
>> At least following new features can be achieved based on uBPF support:
>>
>>   1) Report statistical result:
>>      Like DTrace, perf print statistical report before quit. No need to
>>      extract data using 'perf report'. Statistical method is controled by
>>      user.
>>
>>   2) Control perf's behavior:
>>      Dynamically adjust period of different events. Policy is defined by
>>      user.
>>
>> uBPF library is required before compile. It can be found from github:
>>
>>   https://github.com/iovisor/ubpf.git

[SNIP]

> Interesting!
> If bpf is used for both kernel and user side programs, we can allow
> almost arbitrary C code for the user side.
> There is no need to be limited to a fixed set of helpers.
> There is no verifier in user space either.
> Just call 'printf("string")' directly.

Calling 'printf("string")' would be cool, but it still need
some extra work: .rodata section should be extracted, programs
should be relocated to it.

> Wouldn't even need to change interpreter.
> Also ubpf was written from scratch with apache2, while perf is gpl,
> so you can just link kernel/bpf/core.o directly instead of using external
> libraries.
> I really meant link .o file compiled for kernel.
> Advertize dummy kfree/kmalloc and it will link fine, since perf
> will only be calling __bpf_prog_run() which is 99% indepdendent from kernel.
> I used to do exactly that long ago while performance tunning the interpreter.
> Another option is to fork the interpreter for perf, but I don't like it at all.
> Compiling the same bpf/core.c once for kernel and once for perf is another option,
> but imo linking core.o is easier.

I just realized we can't link apache2 static library into perf. (is that 
true?)

Current perf building doesn't support directly linking like
this, because such linking makes perf rely on kernel building, so we
can't build perf before building kernel any more.

One possible solution: providing a kernel build dir to perf builder
and find the corresponding '.o' file from it:

  $ make KBUILD_DIR=/kernel/build/dir

/lib/`uname -r`/build can be made as default position.

JIT compiler can also be linked this way.

Another possible solution: using macro trick to allow building bpf/core.c
in perf building. It is possible and simpler, but we could be broken by
kernel modification.

> In general this set and overall bpf in user space makes sense only
> if we allow much more flexible C code for user space.
> If it's limited to ubpf_* helpers, that will quickly become suboptimal.

Yes. I tried to reimplement tracex2 in sample but find it is not an easy 
work.
However, in two of my usecase(reporting and controlling), only reporting
require flexible C code. Even we have full featured C, doing statistical 
still
require relative complex code, because the lacking of data structure support
such as associate array (dict in python). For controling (for example, 
dynamically
period adjustments; output perf.data when something unusual detected), uBPF
programs describe rules and invoke actions (actions should be provided 
by perf
helpers), similary to their kernel side counterparts.

So I think no matter uBPF can be 'full C', we should consider making strong
and flexiblity ubpf helpers. Basic reporting method, such as histogram, 
should be
provided by a perf helper directly, no need to be rewritten by uBPF. We
can even make a ubpf helper to bridge uBPF and lua scripts, then invokes 
lua scripts
at perf hooks. For example:

  const char lua_script[] SEC("UBPF-lua;perf_record_exit") "<lua script>";

Thank you.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web