Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1626425 > unrolled thread

Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: net: heap out-of-bounds in  fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone David Ahern <dsa@cumulusnetworks.com> - 2017-04-19 18:10 +0200
    Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone Andrey Konovalov <andreyknvl@google.com> - 2017-04-19 18:20 +0200
      Re: net: heap out-of-bounds in  fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone David Ahern <dsa@cumulusnetworks.com> - 2017-04-19 18:30 +0200
      Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone Cong Wang <xiyou.wangcong@gmail.com> - 2017-04-20 01:50 +0200
        Re: net: heap out-of-bounds in  fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone David Ahern <dsa@cumulusnetworks.com> - 2017-04-20 02:00 +0200
          Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone Dmitry Vyukov <dvyukov@google.com> - 2017-04-20 10:40 +0200
            Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone Andrey Konovalov <andreyknvl@google.com> - 2017-04-20 14:20 +0200
              Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone Andrey Konovalov <andreyknvl@google.com> - 2017-04-20 17:30 +0200
              Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone Andrey Konovalov <andreyknvl@google.com> - 2017-04-20 17:40 +0200
                Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone Andrey Konovalov <andreyknvl@google.com> - 2017-04-20 18:20 +0200
                  Re: net: heap out-of-bounds in  fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone David Ahern <dsa@cumulusnetworks.com> - 2017-04-21 16:30 +0200
                    Re: net: heap out-of-bounds in  fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone David Ahern <dsa@cumulusnetworks.com> - 2017-04-21 20:30 +0200
                    Re: net: heap out-of-bounds in  fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone Eric Dumazet <eric.dumazet@gmail.com> - 2017-04-21 20:50 +0200
              Re: net: heap out-of-bounds in  fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone David Ahern <dsa@cumulusnetworks.com> - 2017-04-20 17:40 +0200

#1626425 — Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone

On 4/18/17 2:43 PM, Andrey Konovalov wrote:
> Hi!
> 
> I've finally managed to reproduce one of the crashes on commit
> 4f7d029b9bf009fbee76bb10c0c4351a1870d2f3 (4.11-rc7).
> 
> I'm not sure if this bug has the same root cause as the first one
> reported in this thread, but it definitely has to do with ipv6
> routing.
> 
> C reproducer, syzkaller program and my .config are attached.

built a kernel with that config. booted the vm. ran the program. nada.

strace is showing:

clone(child_stack=0x72ffb0,
flags=CLONE_NEWUTS|CLONE_NEWUSER|CLONE_NEWPID|CLONE_NEWNET) = -1 EINVAL
(Invalid argument)

[toc] | [next] | [standalone]

#1626449 — Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone

[Multipart message — attachments visible in raw view] — view raw

On Wed, Apr 19, 2017 at 6:09 PM, David Ahern <dsa@cumulusnetworks.com> wrote:
> On 4/18/17 2:43 PM, Andrey Konovalov wrote:
>> Hi!
>>
>> I've finally managed to reproduce one of the crashes on commit
>> 4f7d029b9bf009fbee76bb10c0c4351a1870d2f3 (4.11-rc7).
>>
>> I'm not sure if this bug has the same root cause as the first one
>> reported in this thread, but it definitely has to do with ipv6
>> routing.
>>
>> C reproducer, syzkaller program and my .config are attached.
>
> built a kernel with that config. booted the vm. ran the program. nada.
>
> strace is showing:
>
> clone(child_stack=0x72ffb0,
> flags=CLONE_NEWUTS|CLONE_NEWUSER|CLONE_NEWPID|CLONE_NEWNET) = -1 EINVAL
> (Invalid argument)
>

That's weird. I usually see this when I have CONFIG_USER_NS disabled.

Anyway, I just finished simplifying the reproducer. Give this one a try.

[toc] | [prev] | [next] | [standalone]

#1626479

On 4/19/17 10:12 AM, Andrey Konovalov wrote:
> That's weird. I usually see this when I have CONFIG_USER_NS disabled.

I bungled the movement of .config between servers. reproduced. will
investigate.

[toc] | [prev] | [next] | [standalone]

#1626879 — Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone

On Wed, Apr 19, 2017 at 9:12 AM, Andrey Konovalov <andreyknvl@google.com> wrote:
>
> Anyway, I just finished simplifying the reproducer. Give this one a try.

Thanks for providing such a minimal reproducer!

The following patch could fix this crash, but I am not 100% sure if we should
just clear these bits or reject them with an errno.

diff --git a/net/ipv6/route.c b/net/ipv6/route.c
index 9db14189..cf524c2 100644
--- a/net/ipv6/route.c
+++ b/net/ipv6/route.c
@@ -2086,7 +2086,7 @@ static struct rt6_info
*ip6_route_info_create(struct fib6_config *cfg)
        } else
                rt->rt6i_prefsrc.plen = 0;

-       rt->rt6i_flags = cfg->fc_flags;
+       rt->rt6i_flags = cfg->fc_flags & ~(RTF_PCPU | RTF_CACHE);

 install_route:
        rt->dst.dev = dev;

[toc] | [prev] | [next] | [standalone]

#1626888

On 4/19/17 5:47 PM, Cong Wang wrote:
> On Wed, Apr 19, 2017 at 9:12 AM, Andrey Konovalov <andreyknvl@google.com> wrote:
>>
>> Anyway, I just finished simplifying the reproducer. Give this one a try.
> 
> Thanks for providing such a minimal reproducer!
> 
> The following patch could fix this crash, but I am not 100% sure if we should
> just clear these bits or reject them with an errno.
> 
> diff --git a/net/ipv6/route.c b/net/ipv6/route.c
> index 9db14189..cf524c2 100644
> --- a/net/ipv6/route.c
> +++ b/net/ipv6/route.c
> @@ -2086,7 +2086,7 @@ static struct rt6_info
> *ip6_route_info_create(struct fib6_config *cfg)
>         } else
>                 rt->rt6i_prefsrc.plen = 0;
> 
> -       rt->rt6i_flags = cfg->fc_flags;
> +       rt->rt6i_flags = cfg->fc_flags & ~(RTF_PCPU | RTF_CACHE);
> 
>  install_route:
>         rt->dst.dev = dev;
> 

I sent a patch returning EINVAL if RTF_PCPU is set in fc_flags

[toc] | [prev] | [next] | [standalone]

#1627203 — Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone

On Thu, Apr 20, 2017 at 1:51 AM, David Ahern <dsa@cumulusnetworks.com> wrote:
> On 4/19/17 5:47 PM, Cong Wang wrote:
>> On Wed, Apr 19, 2017 at 9:12 AM, Andrey Konovalov <andreyknvl@google.com> wrote:
>>>
>>> Anyway, I just finished simplifying the reproducer. Give this one a try.
>>
>> Thanks for providing such a minimal reproducer!
>>
>> The following patch could fix this crash, but I am not 100% sure if we should
>> just clear these bits or reject them with an errno.
>>
>> diff --git a/net/ipv6/route.c b/net/ipv6/route.c
>> index 9db14189..cf524c2 100644
>> --- a/net/ipv6/route.c
>> +++ b/net/ipv6/route.c
>> @@ -2086,7 +2086,7 @@ static struct rt6_info
>> *ip6_route_info_create(struct fib6_config *cfg)
>>         } else
>>                 rt->rt6i_prefsrc.plen = 0;
>>
>> -       rt->rt6i_flags = cfg->fc_flags;
>> +       rt->rt6i_flags = cfg->fc_flags & ~(RTF_PCPU | RTF_CACHE);
>>
>>  install_route:
>>         rt->dst.dev = dev;
>>
>
> I sent a patch returning EINVAL if RTF_PCPU is set in fc_flags


Andrey, does it fix the other crashes?

[toc] | [prev] | [next] | [standalone]

#1627404 — Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone

On Thu, Apr 20, 2017 at 10:35 AM, Dmitry Vyukov <dvyukov@google.com> wrote:
> On Thu, Apr 20, 2017 at 1:51 AM, David Ahern <dsa@cumulusnetworks.com> wrote:
>> On 4/19/17 5:47 PM, Cong Wang wrote:
>>> On Wed, Apr 19, 2017 at 9:12 AM, Andrey Konovalov <andreyknvl@google.com> wrote:
>>>>
>>>> Anyway, I just finished simplifying the reproducer. Give this one a try.
>>>
>>> Thanks for providing such a minimal reproducer!
>>>
>>> The following patch could fix this crash, but I am not 100% sure if we should
>>> just clear these bits or reject them with an errno.
>>>
>>> diff --git a/net/ipv6/route.c b/net/ipv6/route.c
>>> index 9db14189..cf524c2 100644
>>> --- a/net/ipv6/route.c
>>> +++ b/net/ipv6/route.c
>>> @@ -2086,7 +2086,7 @@ static struct rt6_info
>>> *ip6_route_info_create(struct fib6_config *cfg)
>>>         } else
>>>                 rt->rt6i_prefsrc.plen = 0;
>>>
>>> -       rt->rt6i_flags = cfg->fc_flags;
>>> +       rt->rt6i_flags = cfg->fc_flags & ~(RTF_PCPU | RTF_CACHE);
>>>
>>>  install_route:
>>>         rt->dst.dev = dev;
>>>
>>
>> I sent a patch returning EINVAL if RTF_PCPU is set in fc_flags
>
>
> Andrey, does it fix the other crashes?

No, still see them.

I'm working on reproducing those.

[toc] | [prev] | [next] | [standalone]

#1627589 — Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone

On Thu, Apr 20, 2017 at 5:28 PM, Andrey Konovalov <andreyknvl@google.com> wrote:
> I've extracted a reproducer for another bug.

It works for me as is, but you might need to run it in a loop.

>
> This one seems to be much closer to what Dmitry reported intially.
>
> ------------[ cut here ]------------
> WARNING: CPU: 1 PID: 3892 at net/ipv6/ip6_fib.c:1472 fib6_del+0xa60/0xdc0
> Modules linked in:
> CPU: 1 PID: 3892 Comm: a.out Not tainted 4.11.0-rc7+ #251
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:16
>  dump_stack+0x292/0x398 lib/dump_stack.c:52
>  __warn+0x19f/0x1e0 kernel/panic.c:549
>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:584
>  fib6_del+0xa60/0xdc0 net/ipv6/ip6_fib.c:1472
>  fib6_clean_node+0x3ce/0x550 net/ipv6/ip6_fib.c:1652
>  fib6_walk_continue+0x577/0x760 net/ipv6/ip6_fib.c:1578
>  fib6_walk+0x91/0xf0 net/ipv6/ip6_fib.c:1623
>  fib6_clean_tree+0x266/0x3c0 net/ipv6/ip6_fib.c:1695
>  __fib6_clean_all+0x1f7/0x3b0 net/ipv6/ip6_fib.c:1711
>  fib6_clean_all+0x27/0x30 net/ipv6/ip6_fib.c:1722
>  rt6_ifdown+0xf7/0x910 net/ipv6/route.c:2820
>  addrconf_ifdown+0x1a3/0x1aa0 net/ipv6/addrconf.c:3541
>  addrconf_notify+0x1bb/0x2570 net/ipv6/addrconf.c:3466
>  notifier_call_chain+0x145/0x2f0 kernel/notifier.c:93
>  __raw_notifier_call_chain kernel/notifier.c:394
>  raw_notifier_call_chain+0x2d/0x40 kernel/notifier.c:401
>  call_netdevice_notifiers_info+0x51/0x90 net/core/dev.c:1647
>  call_netdevice_notifiers net/core/dev.c:1663
>  __dev_notify_flags+0x1fd/0x320 net/core/dev.c:6499
>  dev_change_flags+0xf5/0x140 net/core/dev.c:6530
>  dev_ifsioc+0x62a/0x9f0 net/core/dev_ioctl.c:254
>  dev_ioctl+0x249/0x1160 net/core/dev_ioctl.c:532
>  sock_do_ioctl+0x94/0xb0 net/socket.c:913
>  sock_ioctl+0x28f/0x440 net/socket.c:1004
>  vfs_ioctl fs/ioctl.c:45
>  do_vfs_ioctl+0x1bf/0x1780 fs/ioctl.c:685
>  SYSC_ioctl fs/ioctl.c:700
>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:691
>  entry_SYSCALL_64_fastpath+0x1f/0xc2 arch/x86/entry/entry_64.S:204
> RIP: 0033:0x7f575ded4347
> RSP: 002b:00007fffc94d9948 EFLAGS: 00000217 ORIG_RAX: 0000000000000010
> RAX: ffffffffffffffda RBX: 00007fffc94d9b30 RCX: 00007f575ded4347
> RDX: 00007fffc94d9980 RSI: 0000000000008914 RDI: 0000000000000006
> RBP: 00000000004004e0 R08: 0000000000000028 R09: 0101010101010101
> R10: 0000000000000000 R11: 0000000000000217 R12: 0000000000000000
> R13: 00007fffc94d9b30 R14: 0000000000000000 R15: 0000000000000000
> ---[ end trace 9bda4459ad907043 ]---

[toc] | [prev] | [next] | [standalone]

#1627600 — Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone

On Thu, Apr 20, 2017 at 5:35 PM, David Ahern <dsa@cumulusnetworks.com> wrote:
> On 4/20/17 9:28 AM, Andrey Konovalov wrote:
>> This one seems to be much closer to what Dmitry reported intially.
>
> does not repro here; I ran in a loop and nothing.

You use the attached config, right?

>
> can you send output of "sysctl -a --pattern 'net.ipv6'"

Uploaded here:
https://gist.github.com/xairy/7b6988c9cd8fda5458005df05584ff27

[toc] | [prev] | [next] | [standalone]

#1627627 — Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone

On Thu, Apr 20, 2017 at 5:39 PM, Andrey Konovalov <andreyknvl@google.com> wrote:
> On Thu, Apr 20, 2017 at 5:35 PM, David Ahern <dsa@cumulusnetworks.com> wrote:
>> On 4/20/17 9:28 AM, Andrey Konovalov wrote:
>>> This one seems to be much closer to what Dmitry reported intially.
>>
>> does not repro here; I ran in a loop and nothing.

Here's strace log, maybe it'll help figuring out why it doesn't reproduce:

# strace ./a.out
...
socket(PF_INET6, SOCK_RAW, IPPROTO_TCP) = 3
ioctl(3, SIOCSIFFLAGS, {ifr_name="lo",
ifr_flags=IFF_UP|IFF_PROMISC|IFF_ALLMULTI|0xffff8000}) = 0
socket(PF_INET6, SOCK_RAW|SOCK_CLOEXEC, 0x2b /* IPPROTO_??? */) = 4
socket(PF_INET6, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 5
ioctl(5, SIOCGIFINDEX, {ifr_name="ip6_vti0", ifr_index=26}) = 0
ioctl(4, SIOCSIFADDR, {ifr_name="?", ifr_addr={AF_X25,
"\0\0\32\0\0\0\254*b\333\263\177\0\0"}}) = 0
socket(PF_INET6, SOCK_STREAM|SOCK_NONBLOCK, 0x84 /* IPPROTO_??? */) = 6
ioctl(6, SIOCSIFFLAGS, {ifr_name="lo", ifr_flags=0xffff8000 /* IFF_??? */}) = 0
socket(PF_INET6, SOCK_RAW, 0x3 /* IPPROTO_??? */) = 7
ioctl(7, SIOCSIFFLAGS, {ifr_name="ip6_vti0",
ifr_flags=IFF_UP|IFF_PROMISC|IFF_ALLMULTI|0xffff8000}) = 0
ioctl(3, SIOCSIFFLAGS, {ifr_name="lo",
ifr_flags=IFF_UP|IFF_PROMISC|IFF_ALLMULTI|0xffff8000}) = 0
ioctl(6, SIOCSIFFLAGS, {ifr_name="lo", ifr_flags=0xffff8000 /* IFF_??? */}) = 0
exit_group(0)                           = ?

>
> You use the attached config, right?
>
>>
>> can you send output of "sysctl -a --pattern 'net.ipv6'"
>
> Uploaded here:
> https://gist.github.com/xairy/7b6988c9cd8fda5458005df05584ff27

[toc] | [prev] | [next] | [standalone]

#1628328

On 4/20/17 10:09 AM, Andrey Konovalov wrote:
> On Thu, Apr 20, 2017 at 5:39 PM, Andrey Konovalov <andreyknvl@google.com> wrote:
>> On Thu, Apr 20, 2017 at 5:35 PM, David Ahern <dsa@cumulusnetworks.com> wrote:
>>> On 4/20/17 9:28 AM, Andrey Konovalov wrote:
>>>> This one seems to be much closer to what Dmitry reported intially.
>>> does not repro here; I ran in a loop and nothing.
> Here's strace log, maybe it'll help figuring out why it doesn't reproduce:

reproduced. working on it.

[toc] | [prev] | [next] | [standalone]

#1628429

On 4/21/17 10:47 AM, Eric Dumazet wrote:
> On Fri, 2017-04-21 at 08:27 -0600, David Ahern wrote:
>> On 4/20/17 10:09 AM, Andrey Konovalov wrote:
>>> On Thu, Apr 20, 2017 at 5:39 PM, Andrey Konovalov <andreyknvl@google.com> wrote:
>>>> On Thu, Apr 20, 2017 at 5:35 PM, David Ahern <dsa@cumulusnetworks.com> wrote:
>>>>> On 4/20/17 9:28 AM, Andrey Konovalov wrote:
>>>>>> This one seems to be much closer to what Dmitry reported intially.
>>>>> does not repro here; I ran in a loop and nothing.
>>> Here's strace log, maybe it'll help figuring out why it doesn't reproduce:
>>
>> reproduced. working on it.
> 
> Thanks guys for working on this ;)
> 


Reliable reproducer is the key.

I see what's going on - why the WARN_ON is hit; just looking for the
right fix.

[toc] | [prev] | [next] | [standalone]

#1628484

On Fri, 2017-04-21 at 08:27 -0600, David Ahern wrote:
> On 4/20/17 10:09 AM, Andrey Konovalov wrote:
> > On Thu, Apr 20, 2017 at 5:39 PM, Andrey Konovalov <andreyknvl@google.com> wrote:
> >> On Thu, Apr 20, 2017 at 5:35 PM, David Ahern <dsa@cumulusnetworks.com> wrote:
> >>> On 4/20/17 9:28 AM, Andrey Konovalov wrote:
> >>>> This one seems to be much closer to what Dmitry reported intially.
> >>> does not repro here; I ran in a loop and nothing.
> > Here's strace log, maybe it'll help figuring out why it doesn't reproduce:
> 
> reproduced. working on it.

Thanks guys for working on this ;)

[toc] | [prev] | [next] | [standalone]

#1627601

On 4/20/17 9:28 AM, Andrey Konovalov wrote:
> This one seems to be much closer to what Dmitry reported intially.

does not repro here; I ran in a loop and nothing.

can you send output of "sysctl -a --pattern 'net.ipv6'"

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web