Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1194994 > unrolled thread

Re: llvm bpf debug info. Re: [RFC PATCH v4 3/3] bpf: Introduce function for outputing data to perf event

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: llvm bpf debug info. Re: [RFC PATCH v4 3/3] bpf: Introduce  function for outputing data to perf event He Kuang <hekuang@huawei.com> - 2015-07-29 11:40 +0200
    Re: llvm bpf debug info. Re: [RFC PATCH v4 3/3] bpf: Introduce function for outputing data to perf event pi3orama <pi3orama@163.com> - 2015-07-29 22:10 +0200
    Re: llvm bpf debug info. Re: [RFC PATCH v4 3/3] bpf: Introduce  function for outputing data to perf event Alexei Starovoitov <ast@plumgrid.com> - 2015-08-03 21:50 +0200
      Cc llvmdev: Re: llvm bpf debug info. Re: [RFC PATCH v4 3/3] bpf:  Introduce function for outputing data to perf event "Wangnan (F)" <wangnan0@huawei.com> - 2015-08-04 11:10 +0200

#1194994 — Re: llvm bpf debug info. Re: [RFC PATCH v4 3/3] bpf: Introduce function for outputing data to perf event

Hi, Alexei

On 2015/7/28 10:18, Alexei Starovoitov wrote:
> On 7/25/15 3:04 AM, He Kuang wrote:
>> I noticed that for 64-bit elf format, the reloc sections have
>> 'Addend' in the entry, but there's no 'Addend' info in bpf elf
>> file(64bit). I think there must be something wrong in the process
>> of .s -> .o, which related to 64bit/32bit. Anyway, we can parse out the
>> AT_name now, DW_AT_LOCATION still missed and need your help.
>
> looks like objdump/llvm-dwarfdump can only read known EM,
> but that that shouldn't be the problem for your dwarf reader right?
> It should be able to recognize id-s of ELF::R_X86_64_* relo used right?
> As far as AT_location for testprog.c it seems there is no info for
> local variables because they were optimized away.
> With -O0 I see AT_location being emitted.
> Also line number info seems to be good in both cases.
> But in our case, we don't need this anyway, no? we need to see
> the types of structs mainly or you have some other use cases?
> I think line number info would be great to correlate the error reported
> by verifier into specific line in C.
>

Yes, without AT_location, we can lookup the user output data type
by line number, but there're some issues when we look deep.

There're two steps of work that should be done in user space,
first we embed data type into bpf output record, then we use this
type, or index or some other identifier to lookup the type from
dwarf info, so we got a few plans.

* Plan A. Use line number to identify the user data type

Predefined macros:

   #define DEFINE_BPF_OUTPUT_DATA(type, var)                               \
           const int BPF_OUTPUT_LINE__##var = __LINE__; type var;

   #define BPF_OUTPUT_TRACE_DATA(data, size)       \
           __bpf_output_trace_data(BPF_OUTPUT_LINE__##data, &data, size)

User defined BPF code:

   struct user_define_struct {
          ...
   };

   int testprog(int myvar_a, long myvar_b)
   {
           DEFINE_BPF_OUTPUT_DATA(struct user_define_struct, myvar_c);

           BPF_OUTPUT_TRACE_DATA(myvar_c, sizeof(myvar_c));

   ...

We use macros to embed linenum implicitly, which leads an extra
restriction that user should not define multiple variables in the
same line and not split the macro over multiple lines, like this:

   22 DEFINE_BPF_OUTPUT_DATA(struct xxtype, a); DEFINE_BPF_OUTPUT_DATA(struct xxtype, b);

Or

   22 DEFINE_BPF_OUTPUT_DATA(struct user_define_struct,
   23                               myvar_c);

DW_AT_decl_line = 22, while __LINE__ = 23

So we should add verifier in the llvm BPF backend to warn on the
above codes.

* Plan B. Lookup variable type from dwarf AT_location info

We can make use of the output data variable's address, for bpf is
a minus offset to frame base. Then lookup matched offset from
location info(e.g. "DW_OP_fbreg: -32") to identify the variable
type.

For getting the frame base address, we can use builtin functions
like __builtin_frame_base() and __builtin_dwarf_cfa() which
returns the call frame base address. Currently those builtin
functions are not implemented in BPF lower operation yet, so we
tested our bpf program by using a variable tag on frame base, as
following:

   struct user_define_struct {
      ...
   };

   typedef struct {} frame_base_tag;

   int testprog(void)
   {
       frame_base_tag BPF_FRAME_BASE;
       struct user_define_struct myvar_a;

       __bpf_trace_output_data((void *)&myvar_a - (void *)&BPF_FRAME_BASE,
                               &myvar_a, sizeof(myvar_a));
   ...

The first argument of __bpf_trace_output_data() will be caculated
and it's easy to traverse the variable DIEs in dwarf info and
check each DW_AT_location attribute to find the corresponding
variable type.

The things let us worry about is the opimization may reuse the
stack space which can cause different variables share the same
address, by some rough tests that kind of optimization does not
appear.

* Comparison

Plan A needs less effort and easy to implement, but requires more
check to ensure user not use multiple definition in the same line
and not use macro cross lines.

The advantages of plan B is that we do not need introduce macros
showed in above example and all the things are done implicitly,
but the AT_location info is the prerequisite of this plan, I'm
not sure whether we can guarantee this info in dwarf or not.

Another way we can think of is adding new builtin functions to
indicate the compilier to generate codes return the dwarf type
index directly:

   __bpf_trace_output_data(__builtin_dwarf_type(myvar_a), &myvar_a, size);

What's your opinion on those plans, and do you have more
suggestion?

Thank you.




--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1195475 — Re: llvm bpf debug info. Re: [RFC PATCH v4 3/3] bpf: Introduce function for outputing data to perf event

发自我的 iPhone

> 在 2015年7月30日，上午1:13，Alexei Starovoitov <ast@plumgrid.com> 写道：
> 
>> On 7/29/15 2:38 AM, He Kuang wrote:
>> Hi, Alexei
>> 
>>> On 2015/7/28 10:18, Alexei Starovoitov wrote:
>>>> On 7/25/15 3:04 AM, He Kuang wrote:
>>>> I noticed that for 64-bit elf format, the reloc sections have
>>>> 'Addend' in the entry, but there's no 'Addend' info in bpf elf
>>>> file(64bit). I think there must be something wrong in the process
>>>> of .s -> .o, which related to 64bit/32bit. Anyway, we can parse out the
>>>> AT_name now, DW_AT_LOCATION still missed and need your help.
>>> 
>>> looks like objdump/llvm-dwarfdump can only read known EM,
>>> but that that shouldn't be the problem for your dwarf reader right?
>>> It should be able to recognize id-s of ELF::R_X86_64_* relo used right?
>>> As far as AT_location for testprog.c it seems there is no info for
>>> local variables because they were optimized away.
>>> With -O0 I see AT_location being emitted.
>>> Also line number info seems to be good in both cases.
>>> But in our case, we don't need this anyway, no? we need to see
>>> the types of structs mainly or you have some other use cases?
>>> I think line number info would be great to correlate the error reported
>>> by verifier into specific line in C.
>> 
>> Yes, without AT_location, we can lookup the user output data type
>> by line number, but there're some issues when we look deep.
>> 
>> There're two steps of work that should be done in user space,
>> first we embed data type into bpf output record, then we use this
>> type, or index or some other identifier to lookup the type from
>> dwarf info, so we got a few plans.
>> 
>> * Plan A. Use line number to identify the user data type
>> 
>> Predefined macros:
>> 
>>   #define DEFINE_BPF_OUTPUT_DATA(type,
>> var)                               \
>>           const int BPF_OUTPUT_LINE__##var = __LINE__; type var;
>> 
>>   #define BPF_OUTPUT_TRACE_DATA(data, size)       \
>>           __bpf_output_trace_data(BPF_OUTPUT_LINE__##data, &data, size)
>> 
>> User defined BPF code:
>> 
>>   struct user_define_struct {
>>          ...
>>   };
>> 
>>   int testprog(int myvar_a, long myvar_b)
>>   {
>>           DEFINE_BPF_OUTPUT_DATA(struct user_define_struct, myvar_c);
>> 
>>           BPF_OUTPUT_TRACE_DATA(myvar_c, sizeof(myvar_c));
>> 
>>   ...
>> 
>> We use macros to embed linenum implicitly, which leads an extra
>> restriction that user should not define multiple variables in the
>> same line and not split the macro over multiple lines, like this:
>> 
>>   22 DEFINE_BPF_OUTPUT_DATA(struct xxtype, a);
>> DEFINE_BPF_OUTPUT_DATA(struct xxtype, b);
>> 
>> Or
>> 
>>   22 DEFINE_BPF_OUTPUT_DATA(struct user_define_struct,
>>   23                               myvar_c);
>> 
>> DW_AT_decl_line = 22, while __LINE__ = 23
>> 
>> So we should add verifier in the llvm BPF backend to warn on the
>> above codes.
>> 
>> * Plan B. Lookup variable type from dwarf AT_location info
>> 
>> We can make use of the output data variable's address, for bpf is
>> a minus offset to frame base. Then lookup matched offset from
>> location info(e.g. "DW_OP_fbreg: -32") to identify the variable
>> type.
>> 
>> For getting the frame base address, we can use builtin functions
>> like __builtin_frame_base() and __builtin_dwarf_cfa() which
>> returns the call frame base address. Currently those builtin
>> functions are not implemented in BPF lower operation yet, so we
>> tested our bpf program by using a variable tag on frame base, as
>> following:
>> 
>>   struct user_define_struct {
>>      ...
>>   };
>> 
>>   typedef struct {} frame_base_tag;
>> 
>>   int testprog(void)
>>   {
>>       frame_base_tag BPF_FRAME_BASE;
>>       struct user_define_struct myvar_a;
>> 
>>       __bpf_trace_output_data((void *)&myvar_a - (void *)&BPF_FRAME_BASE,
>>                               &myvar_a, sizeof(myvar_a));
>>   ...
>> 
>> The first argument of __bpf_trace_output_data() will be caculated
>> and it's easy to traverse the variable DIEs in dwarf info and
>> check each DW_AT_location attribute to find the corresponding
>> variable type.
>> 
>> The things let us worry about is the opimization may reuse the
>> stack space which can cause different variables share the same
>> address, by some rough tests that kind of optimization does not
>> appear.
>> 
>> * Comparison
>> 
>> Plan A needs less effort and easy to implement, but requires more
>> check to ensure user not use multiple definition in the same line
>> and not use macro cross lines.
>> 
>> The advantages of plan B is that we do not need introduce macros
>> showed in above example and all the things are done implicitly,
>> but the AT_location info is the prerequisite of this plan, I'm
>> not sure whether we can guarantee this info in dwarf or not.
>> 
>> Another way we can think of is adding new builtin functions to
>> indicate the compilier to generate codes return the dwarf type
>> index directly:
>> 
>>   __bpf_trace_output_data(__builtin_dwarf_type(myvar_a), &myvar_a, size);
> 
> probably both A and B won't really work when programs get bigger
> and optimizations will start moving lines around.
> the builtin_dwarf_type idea is actually quite interesting.
> Potentially that builtin can stringify type name and later we can
> search it in dwarf. Please take a look how to add such builtin.
> There are few similar builtins that deal with exception handling
> and need type info. May be they can be reused. Like:
> int_eh_typeid_for and int_eh_dwarf_cfa
> 

Hi Alexei,

I was wondering if you could give us a hint on adding BPF specific builtins?

Doesn't like other machines, currently there's no Builtins.def for BPF to hold
builtins for that specific target. If we start creating such builtins, we can bring
more there. One builtins I want to see should be __builtin_bpf_strcmp(char*, char*),
with that we can filter events base on name of a task. What I really need is filtering 
events based on comm of the main thread, which should be useful on Android 
since in Android name of working threads are always something like "Binder_?", 
only the main threads names are meaningful. But let's work on strcmp first.

Thank you.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1199229

On 7/31/15 3:18 AM, Wangnan (F) wrote:
>
>     However with the newest llvm + clang the DWARF info is still incorrect:
>
> $ objdump  --dwarf=info ./out.o
> ...
>   <1><3f>: Abbrev Number: 3 (DW_TAG_structure_type)
>      <40>   DW_AT_name        : (indirect string, offset: 0x0): clang
> version 3.8.0 (http://llvm.org/git/clang.git
> f0fcd3432cbed83500df70c18f275d8affb89e5e) (http://llvm.org/git/llvm.git
> c8ccd78d31d4949fa1c14e954ccb06253e18cf37)
>      <44>   DW_AT_byte_size   : 8
>      <45>   DW_AT_decl_file   : 1
>      <46>   DW_AT_decl_line   : 4
>   <2><47>: Abbrev Number: 4 (DW_TAG_member)
>      <48>   DW_AT_name        : (indirect string, offset: 0x0): clang
> version 3.8.0 (http://llvm.org/git/clang.git
> f0fcd3432cbed83500df70c18f275d8affb89e5e) (http://llvm.org/git/llvm.git
> c8ccd78d31d4949fa1c14e954ccb06253e18cf37)
>      <4c>   DW_AT_type        : <0x60>
>      <50>   DW_AT_decl_file   : 1
>      <51>   DW_AT_decl_line   : 5
>      <52>   DW_AT_data_member_location: 0
>   <2><53>: Abbrev Number: 4 (DW_TAG_member)
>      <54>   DW_AT_name        : (indirect string, offset: 0x0): clang
> version 3.8.0 (http://llvm.org/git/clang.git
> f0fcd3432cbed83500df70c18f275d8affb89e5e) (http://llvm.org/git/llvm.git
> c8ccd78d31d4949fa1c14e954ccb06253e18cf37)
>      <58>   DW_AT_type        : <0x60>
>      <5c>   DW_AT_decl_file   : 1
>      <5d>   DW_AT_decl_line   : 6
>      <5e>   DW_AT_data_member_location: 4
> ...
>
> The DW_AT_name is missing.

didn't have time to look at it.
from your llvm patches looks like you've got quite experienced
with it already :)

> I'll post 2 LLVM patches by replying this mail. Please have a look and
> help me
> send them to LLVM if you think my code is correct.

patch 1:
I don't quite understand the purpose of builtin_dwarf_cfa
returning R11. It's a special register seen inside llvm codegen
only. It doesn't have kernel meaning.

patch 2:
do we really need to hack clang?
Can you just define a function that aliases to intrinsic,
like we do for ld_abs/ld_ind ?
void bpf_store_half(void *skb, u64 off, u64 val) asm("llvm.bpf.store.half");
then no extra patches necessary.

> struct my_str {
>          int x;
>          int y;
> };
> struct my_str __str_my_str;
>
> struct my_str2 {
>          int x;
>          int y;
>          int z;
> };
> struct my_str2 __str_my_str2;
>
>          test_func(__builtin_bpf_typeid(&__str_my_str));
>          test_func(__builtin_bpf_typeid(&__str_my_str2));
>          mov     r1, 1
>          call    4660
>          mov     r1, 2
>          call    4660

this part looks good. I think it's usable.

 > 1. llvm.eh_typeid_for can be used on global variables only. So for each
 > output
 >     structure we have to define a global varable.

why? I think it should work with local pointers as well.

 > 2. We still need to find a way to connect the fetchd typeid with DWARF
 > info.
 >     Inserting that ID into DWARF may workable?

hmm, that id should be the same id we're seeing in dwarf, right?
I think it's used in exception handling which is reusing some of
the dwarf stuff for this, so the must be a way to connect that id
to actual type info. Though last time I looked at EH was
during g++ hacking days. No idea how llvm does it exactly, but
I'm assuming the logic for rtti should be similar.

btw, for any deep llvm stuff you may need to move the thread to llvmdev.
May be folks there will have other ideas.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1199571 — Cc llvmdev: Re: llvm bpf debug info. Re: [RFC PATCH v4 3/3] bpf: Introduce function for outputing data to perf event

For people who in llvmdev:

This mail is belong to a thread in linux kernel mailing list, the first 
message
can be retrived from:

  http://lkml.kernel.org/r/55B1535E.8090406@plumgrid.com

Our goal is to fild a way to make BPF program get an unique ID for each type
so it can pass the ID to other part of kernel, then we can retrive the 
type and
decode the structure using DWARF information. Currently we have two problem
needs to solve:

1. Dwarf information generated by BPF backend lost all DW_AT_name field;

2. How to get typeid from local variable? I tried llvm.eh_typeid_for
    but it support global variable only.

Following is my response to Alexei.

On 2015/8/4 3:44, Alexei Starovoitov wrote:
> On 7/31/15 3:18 AM, Wangnan (F) wrote:
>

[SNIP]

> didn't have time to look at it.
> from your llvm patches looks like you've got quite experienced
> with it already :)
>
>> I'll post 2 LLVM patches by replying this mail. Please have a look and
>> help me
>> send them to LLVM if you think my code is correct.
>
> patch 1:
> I don't quite understand the purpose of builtin_dwarf_cfa
> returning R11. It's a special register seen inside llvm codegen
> only. It doesn't have kernel meaning.
>

Kernel side verifier allows us to do arithmetic computation using two 
local variable
address or local variable address and R11. Therefore, we can compute the 
location
of a local variable using:

   mark = &my_var_a - __builtin_frame_address(0);

If the stack allocation is fixed (if the location is never reused), the 
above 'mark'
can be uniquely identify a local variable. That's why I'm interesting in 
it. However
I'm not sure whether the prerequestion is hold.

> patch 2:
> do we really need to hack clang?
> Can you just define a function that aliases to intrinsic,
> like we do for ld_abs/ld_ind ?
> void bpf_store_half(void *skb, u64 off, u64 val) 
> asm("llvm.bpf.store.half");
> then no extra patches necessary.
>
>> struct my_str {
>>          int x;
>>          int y;
>> };
>> struct my_str __str_my_str;
>>
>> struct my_str2 {
>>          int x;
>>          int y;
>>          int z;
>> };
>> struct my_str2 __str_my_str2;
>>
>>          test_func(__builtin_bpf_typeid(&__str_my_str));
>>          test_func(__builtin_bpf_typeid(&__str_my_str2));
>>          mov     r1, 1
>>          call    4660
>>          mov     r1, 2
>>          call    4660
>
> this part looks good. I think it's usable.
>
> > 1. llvm.eh_typeid_for can be used on global variables only. So for each
> > output
> >     structure we have to define a global varable.
>
> why? I think it should work with local pointers as well.
>

It is defined by LLVM, in lib/CodeGen/Analysis.cpp:

/// ExtractTypeInfo - Returns the type info, possibly bitcast, encoded in V.
GlobalValue *llvm::ExtractTypeInfo(Value *V) {
   ...
   assert((GV || isa<ConstantPointerNull>(V)) &&
          "TypeInfo must be a global variable or NULL");   <-- we can 
use only constant pointers
   return GV;
}

So from llvm::Intrinsic::eh_typeid_for we can get type of global 
variables only.

We may need a new intrinsic for that.


> > 2. We still need to find a way to connect the fetchd typeid with DWARF
> > info.
> >     Inserting that ID into DWARF may workable?
>
> hmm, that id should be the same id we're seeing in dwarf, right?

There's no 'typeid' field in dwarf originally. I'm still looking for a way
to inject this ID into dwarf infromation.

> I think it's used in exception handling which is reusing some of
> the dwarf stuff for this, so the must be a way to connect that id
> to actual type info. Though last time I looked at EH was
> during g++ hacking days. No idea how llvm does it exactly, but
> I'm assuming the logic for rtti should be similar.
>

I'm not sure whether RTTI use dwarf to deduce type information. I think not,
because dwarf infos can be stripped out.

> btw, for any deep llvm stuff you may need to move the thread to llvmdev.
> May be folks there will have other ideas.
>


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web