Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1715753 > unrolled thread

Re: [PATCH net-next v3] arm: eBPF JIT compiler

Started byShubham Bansal <illusionist.neo@gmail.com>
First post2017-08-19 22:00 +0200
Last post2017-08-19 23:50 +0200
Articles 4 — 1 participant

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCH net-next v3] arm: eBPF JIT compiler Shubham Bansal <illusionist.neo@gmail.com> - 2017-08-19 22:00 +0200
    Re: [PATCH net-next v3] arm: eBPF JIT compiler Shubham Bansal <illusionist.neo@gmail.com> - 2017-08-19 22:10 +0200
    Re: [PATCH net-next v3] arm: eBPF JIT compiler Shubham Bansal <illusionist.neo@gmail.com> - 2017-08-19 23:30 +0200
      Re: [PATCH net-next v3] arm: eBPF JIT compiler Shubham Bansal <illusionist.neo@gmail.com> - 2017-08-19 23:50 +0200

#1715753 — Re: [PATCH net-next v3] arm: eBPF JIT compiler

FromShubham Bansal <illusionist.neo@gmail.com>
Date2017-08-19 22:00 +0200
SubjectRe: [PATCH net-next v3] arm: eBPF JIT compiler
Message-ID<ugi4x-5rv-1@gated-at.bofh.it>
> impressive work.
> Acked-by: Alexei Starovoitov <ast@kernel.org>

Thanks :)

I can't take all the credit. It was Daniel and Kees who helped me a lot.
I would have given up a long time ago without them.
>
> Any performance numbers with vs without JIT ?

Here is the mail from Kees on v1 of the patch.

For what it's worth, I did an comparison of the numbers Shubham posted
in another thread for the JIT, comparing the eBPF interpreter with his
new JIT. The post is here:

https://www.spinics.net/lists/netdev/msg436402.html

Other than that I can send the test runs which have time, but I will
not be able to compare them like kees this week.
Does that sound good?
>
>> +static const u8 bpf2a32[][2] = {
>> +       /* return value from in-kernel function, and exit value from eBPF
>> */
>> +       [BPF_REG_0] = {ARM_R1, ARM_R0},
>> +       /* arguments from eBPF program to in-kernel function */
>> +       [BPF_REG_1] = {ARM_R3, ARM_R2},
>
>
> as far as i understand arm32 calling convention the mapping makes sense
> to me. Hard to come up with anything better than the above.
I tried different versions of it, according to the need of different
eBPF instructions, as you can see, we are register deficient. This is
the best I could come up with.
Would love to hear any improvement over this.
>
>> +       /* function call */
>> +       case BPF_JMP | BPF_CALL:
>> +       {
>> +               const u8 *r0 = bpf2a32[BPF_REG_0];
>> +               const u8 *r1 = bpf2a32[BPF_REG_1];
>> +               const u8 *r2 = bpf2a32[BPF_REG_2];
>> +               const u8 *r3 = bpf2a32[BPF_REG_3];
>> +               const u8 *r4 = bpf2a32[BPF_REG_4];
>> +               const u8 *r5 = bpf2a32[BPF_REG_5];
>> +               const u32 func = (u32)__bpf_call_base + (u32)imm;
>> +
>> +               emit_a32_mov_r64(true, r0, r1, false, false, ctx);
>> +               emit_a32_mov_r64(true, r1, r2, false, true, ctx);
>> +               emit_push_r64(r5, 0, ctx);
>> +               emit_push_r64(r4, 8, ctx);
>> +               emit_push_r64(r3, 16, ctx);
>> +
>> +               emit_a32_mov_i(tmp[1], func, false, ctx);
>> +               emit_blx_r(tmp[1], ctx);
>
>
> to improve the cost of call we can teach verifier to mark the registers
> actually used to pass arguments, so not all pushes would be needed.
> But it may be drop in the bucket comparing to the cost of compound
> 64-bit alu ops.
Thats right. But still an improvement I guess. I think I discussed it
with Daniel and I thought, I should get this patch reach mainstream
first then I can improve on it.
> There was some work on llvm side to use 32-bit subregisters which
> should help 32-bit architectures and JITs, but it didn't go far.
> So if you're interested further improving bpf program speeds on arm32
> you may take a look at llvm side. I can certainly provide the tips.
Sure. Sounds good.

Best,
Shubham

[toc] | [next] | [standalone]

#1715756

FromShubham Bansal <illusionist.neo@gmail.com>
Date2017-08-19 22:10 +0200
Message-ID<ugiee-5LF-11@gated-at.bofh.it>
In reply to#1715753
One more thing I forgot to mention.

I think this is the first implementation of eBPF JIT on any 32 bit
arch, correct me if I am wrong. I think we can use this as a POC to
implement eBPF on other 32 bit arch as well like x86, depends on its
need I guess.

[toc] | [prev] | [next] | [standalone]

#1715768

FromShubham Bansal <illusionist.neo@gmail.com>
Date2017-08-19 23:30 +0200
Message-ID<ugjtE-6pT-5@gated-at.bofh.it>
In reply to#1715753
Here are numbers.

Without any JIT enabled

test_pkt_access:PASS:ipv4 1823 nsec
test_pkt_access:PASS:ipv6 1743 nsec
test_xdp:PASS:ipv4 769022 nsec
test_xdp:PASS:ipv6 15408 nsec
test_l4lb:PASS:ipv4 12441 nsec
test_l4lb:PASS:ipv6 18131 nsec
test_tcp_estats:PASS: 0 nsec
test_bpf_obj_id:PASS:get-fd-by-notexist-prog-id 0 nsec
test_bpf_obj_id:PASS:get-fd-by-notexist-map-id 0 nsec
test_bpf_obj_id:PASS:get-prog-info(fd) 0 nsec
test_bpf_obj_id:PASS:get-map-info(fd) 0 nsec
test_bpf_obj_id:PASS:get-prog-info(fd) 0 nsec
test_bpf_obj_id:PASS:get-map-info(fd) 0 nsec
test_bpf_obj_id:PASS:get-prog-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-prog-info(next_id->fd) 0 nsec
test_bpf_obj_id:PASS:get-prog-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-prog-info(next_id->fd) 0 nsec
test_bpf_obj_id:PASS:check total prog id found by get_next_id 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:check get-map-info(next_id->fd) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:check get-map-info(next_id->fd) 0 nsec
test_bpf_obj_id:PASS:check total map id found by get_next_id 0 nsec
test_pkt_md_access:PASS: 5816 nsec
Summary: 30 PASSED, 0 FAILED

With only bpf_jit_enable

test_pkt_access:PASS:ipv4 263 nsec
test_pkt_access:PASS:ipv6 337 nsec
test_xdp:PASS:ipv4 1377212 nsec
test_xdp:PASS:ipv6 1290381 nsec
test_l4lb:PASS:ipv4 2689 nsec
test_l4lb:PASS:ipv6 3804 nsec
test_tcp_estats:PASS: 0 nsec
test_bpf_obj_id:PASS:get-fd-by-notexist-prog-id 0 nsec
test_bpf_obj_id:PASS:get-fd-by-notexist-map-id 0 nsec
test_bpf_obj_id:PASS:get-prog-info(fd) 0 nsec
test_bpf_obj_id:PASS:get-map-info(fd) 0 nsec
test_bpf_obj_id:PASS:get-prog-info(fd) 0 nsec
test_bpf_obj_id:PASS:get-map-info(fd) 0 nsec
test_bpf_obj_id:PASS:get-prog-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-prog-info(next_id->fd) 0 nsec
test_bpf_obj_id:PASS:get-prog-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-prog-info(next_id->fd) 0 nsec
test_bpf_obj_id:PASS:check total prog id found by get_next_id 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:check get-map-info(next_id->fd) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:check get-map-info(next_id->fd) 0 nsec
test_bpf_obj_id:PASS:check total map id found by get_next_id 0 nsec
test_pkt_md_access:PASS: 41609 nsec
Summary: 30 PASSED, 0 FAILED


With bpf_jit_enable and bpf_jit_harden set

test_pkt_access:PASS:ipv4 416 nsec
test_pkt_access:PASS:ipv6 509 nsec
test_xdp:PASS:ipv4 1512428 nsec
test_xdp:PASS:ipv6 962318 nsec
test_l4lb:PASS:ipv4 4135 nsec
test_l4lb:PASS:ipv6 5497 nsec
test_tcp_estats:PASS: 0 nsec
test_bpf_obj_id:PASS:get-fd-by-notexist-prog-id 0 nsec
test_bpf_obj_id:PASS:get-fd-by-notexist-map-id 0 nsec
test_bpf_obj_id:PASS:get-prog-info(fd) 0 nsec
test_bpf_obj_id:PASS:get-map-info(fd) 0 nsec
test_bpf_obj_id:PASS:get-prog-info(fd) 0 nsec
test_bpf_obj_id:PASS:get-map-info(fd) 0 nsec
test_bpf_obj_id:PASS:get-prog-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-prog-info(next_id->fd) 0 nsec
test_bpf_obj_id:PASS:get-prog-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-prog-info(next_id->fd) 0 nsec
test_bpf_obj_id:PASS:check total prog id found by get_next_id 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:check get-map-info(next_id->fd) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:check get-map-info(next_id->fd) 0 nsec
test_bpf_obj_id:PASS:check total map id found by get_next_id 0 nsec
test_pkt_md_access:PASS: 24774 nsec
Summary: 30 PASSED, 0 FAILED

-Shubham

[toc] | [prev] | [next] | [standalone]

#1715770

FromShubham Bansal <illusionist.neo@gmail.com>
Date2017-08-19 23:50 +0200
Message-ID<ugjN0-6xc-5@gated-at.bofh.it>
In reply to#1715768
Qemu is giving me different numbers different time.
Another stats.

With bpf_jit_enable set

test_pkt_access:PASS:ipv4 271 nsec
test_pkt_access:PASS:ipv6 297 nsec
test_xdp:PASS:ipv4 961517 nsec     <--- Here is the difference.
test_xdp:PASS:ipv6 615855 nsec     <--- Here is the difference.
test_l4lb:PASS:ipv4 3049 nsec
test_l4lb:PASS:ipv6 3906 nsec
test_tcp_estats:PASS: 0 nsec
test_bpf_obj_id:PASS:get-fd-by-notexist-prog-id 0 nsec
test_bpf_obj_id:PASS:get-fd-by-notexist-map-id 0 nsec
test_bpf_obj_id:PASS:get-prog-info(fd) 0 nsec
test_bpf_obj_id:PASS:get-map-info(fd) 0 nsec
test_bpf_obj_id:PASS:get-prog-info(fd) 0 nsec
test_bpf_obj_id:PASS:get-map-info(fd) 0 nsec
test_bpf_obj_id:PASS:get-prog-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-prog-info(next_id->fd) 0 nsec
test_bpf_obj_id:PASS:get-prog-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-prog-info(next_id->fd) 0 nsec
test_bpf_obj_id:PASS:check total prog id found by get_next_id 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:check get-map-info(next_id->fd) 0 nsec
test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
test_bpf_obj_id:PASS:check get-map-info(next_id->fd) 0 nsec
test_bpf_obj_id:PASS:check total map id found by get_next_id 0 nsec
test_pkt_md_access:PASS: 30459 nsec    <--- Here is the difference.
Summary: 30 PASSED, 0 FAILED


On Sun, Aug 20, 2017 at 2:58 AM, Shubham Bansal
<illusionist.neo@gmail.com> wrote:
> Here are numbers.
>
> Without any JIT enabled
>
> test_pkt_access:PASS:ipv4 1823 nsec
> test_pkt_access:PASS:ipv6 1743 nsec
> test_xdp:PASS:ipv4 769022 nsec
> test_xdp:PASS:ipv6 15408 nsec
> test_l4lb:PASS:ipv4 12441 nsec
> test_l4lb:PASS:ipv6 18131 nsec
> test_tcp_estats:PASS: 0 nsec
> test_bpf_obj_id:PASS:get-fd-by-notexist-prog-id 0 nsec
> test_bpf_obj_id:PASS:get-fd-by-notexist-map-id 0 nsec
> test_bpf_obj_id:PASS:get-prog-info(fd) 0 nsec
> test_bpf_obj_id:PASS:get-map-info(fd) 0 nsec
> test_bpf_obj_id:PASS:get-prog-info(fd) 0 nsec
> test_bpf_obj_id:PASS:get-map-info(fd) 0 nsec
> test_bpf_obj_id:PASS:get-prog-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:get-prog-info(next_id->fd) 0 nsec
> test_bpf_obj_id:PASS:get-prog-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:get-prog-info(next_id->fd) 0 nsec
> test_bpf_obj_id:PASS:check total prog id found by get_next_id 0 nsec
> test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:check get-map-info(next_id->fd) 0 nsec
> test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:check get-map-info(next_id->fd) 0 nsec
> test_bpf_obj_id:PASS:check total map id found by get_next_id 0 nsec
> test_pkt_md_access:PASS: 5816 nsec
> Summary: 30 PASSED, 0 FAILED
>
> With only bpf_jit_enable
>
> test_pkt_access:PASS:ipv4 263 nsec
> test_pkt_access:PASS:ipv6 337 nsec
> test_xdp:PASS:ipv4 1377212 nsec
> test_xdp:PASS:ipv6 1290381 nsec
> test_l4lb:PASS:ipv4 2689 nsec
> test_l4lb:PASS:ipv6 3804 nsec
> test_tcp_estats:PASS: 0 nsec
> test_bpf_obj_id:PASS:get-fd-by-notexist-prog-id 0 nsec
> test_bpf_obj_id:PASS:get-fd-by-notexist-map-id 0 nsec
> test_bpf_obj_id:PASS:get-prog-info(fd) 0 nsec
> test_bpf_obj_id:PASS:get-map-info(fd) 0 nsec
> test_bpf_obj_id:PASS:get-prog-info(fd) 0 nsec
> test_bpf_obj_id:PASS:get-map-info(fd) 0 nsec
> test_bpf_obj_id:PASS:get-prog-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:get-prog-info(next_id->fd) 0 nsec
> test_bpf_obj_id:PASS:get-prog-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:get-prog-info(next_id->fd) 0 nsec
> test_bpf_obj_id:PASS:check total prog id found by get_next_id 0 nsec
> test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:check get-map-info(next_id->fd) 0 nsec
> test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:check get-map-info(next_id->fd) 0 nsec
> test_bpf_obj_id:PASS:check total map id found by get_next_id 0 nsec
> test_pkt_md_access:PASS: 41609 nsec
> Summary: 30 PASSED, 0 FAILED
>
>
> With bpf_jit_enable and bpf_jit_harden set
>
> test_pkt_access:PASS:ipv4 416 nsec
> test_pkt_access:PASS:ipv6 509 nsec
> test_xdp:PASS:ipv4 1512428 nsec
> test_xdp:PASS:ipv6 962318 nsec
> test_l4lb:PASS:ipv4 4135 nsec
> test_l4lb:PASS:ipv6 5497 nsec
> test_tcp_estats:PASS: 0 nsec
> test_bpf_obj_id:PASS:get-fd-by-notexist-prog-id 0 nsec
> test_bpf_obj_id:PASS:get-fd-by-notexist-map-id 0 nsec
> test_bpf_obj_id:PASS:get-prog-info(fd) 0 nsec
> test_bpf_obj_id:PASS:get-map-info(fd) 0 nsec
> test_bpf_obj_id:PASS:get-prog-info(fd) 0 nsec
> test_bpf_obj_id:PASS:get-map-info(fd) 0 nsec
> test_bpf_obj_id:PASS:get-prog-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:get-prog-info(next_id->fd) 0 nsec
> test_bpf_obj_id:PASS:get-prog-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:get-prog-info(next_id->fd) 0 nsec
> test_bpf_obj_id:PASS:check total prog id found by get_next_id 0 nsec
> test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:check get-map-info(next_id->fd) 0 nsec
> test_bpf_obj_id:PASS:get-map-fd(next_id) 0 nsec
> test_bpf_obj_id:PASS:check get-map-info(next_id->fd) 0 nsec
> test_bpf_obj_id:PASS:check total map id found by get_next_id 0 nsec
> test_pkt_md_access:PASS: 24774 nsec
> Summary: 30 PASSED, 0 FAILED
>
> -Shubham

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web