Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1592166 > unrolled thread

net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone

Back to article view | Back to linux.kernel

Contents

  net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone Dmitry Vyukov <dvyukov@google.com> - 2017-03-03 18:40 +0100
    Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone Dmitry Vyukov <dvyukov@google.com> - 2017-03-03 20:20 +0100
    Re: net: heap out-of-bounds in  fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone David Ahern <dsa@cumulusnetworks.com> - 2017-03-03 20:50 +0100
      Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone Dmitry Vyukov <dvyukov@google.com> - 2017-03-04 20:00 +0100
        Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone Dmitry Vyukov <dvyukov@google.com> - 2017-03-04 20:10 +0100
        Re: net: heap out-of-bounds in  fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone Eric Dumazet <eric.dumazet@gmail.com> - 2017-03-04 21:20 +0100
          Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone Dmitry Vyukov <dvyukov@google.com> - 2017-03-05 12:00 +0100
          Re: net: heap out-of-bounds in  fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone David Ahern <dsa@cumulusnetworks.com> - 2017-03-06 18:40 +0100
            Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone Dmitry Vyukov <dvyukov@google.com> - 2017-03-06 20:00 +0100
              Re: net: heap out-of-bounds in  fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone David Ahern <dsa@cumulusnetworks.com> - 2017-03-07 01:00 +0100
                Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone Dmitry Vyukov <dvyukov@google.com> - 2017-03-07 09:50 +0100
                  Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone Dmitry Vyukov <dvyukov@google.com> - 2017-03-07 11:30 +0100
                    Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone Dmitry Vyukov <dvyukov@google.com> - 2017-03-07 19:20 +0100
                      Re: net: heap out-of-bounds in  fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone David Ahern <dsa@cumulusnetworks.com> - 2017-03-07 19:50 +0100
                        Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone Dmitry Vyukov <dvyukov@google.com> - 2017-03-07 20:20 +0100
                          Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone Dmitry Vyukov <dvyukov@google.com> - 2017-03-07 20:50 +0100
                            Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone Dmitry Vyukov <dvyukov@google.com> - 2017-03-07 21:10 +0100
                              Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone Dmitry Vyukov <dvyukov@google.com> - 2017-03-08 13:30 +0100
                    Re: net: heap out-of-bounds in  fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone David Ahern <dsa@cumulusnetworks.com> - 2017-03-08 05:50 +0100
                  Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone Dmitry Vyukov <dvyukov@google.com> - 2017-03-07 18:50 +0100
                  Re: net: heap out-of-bounds in  fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone David Ahern <dsa@cumulusnetworks.com> - 2017-03-07 18:50 +0100
                  Re: net: heap out-of-bounds in  fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone David Ahern <dsa@cumulusnetworks.com> - 2017-03-07 19:30 +0100

Page 1 of 2  [1] 2  Next page →

#1592166 — net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone

Hello,

I am getting heap out-of-bounds reports in
fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone while running
syzkaller fuzzer on 86292b33d4b79ee03e2f43ea0381ef85f077c760. They all
follow the same pattern: an object of size 216 is allocated from
ip_dst_cache slab, and then accessed at offset 272/276 withing
fib6_walk. Looks like type confusion. Unfortunately this is not
reproducible.

==================================================================
BUG: KASAN: slab-out-of-bounds in rt6_dump_route+0x293/0x2f0
net/ipv6/route.c:3547 at addr ffff88004b864514
Read of size 4 by task syz-executor7/25042
CPU: 0 PID: 25042 Comm: syz-executor7 Not tainted 4.10.0+ #234
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:15 [inline]
 dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
 kasan_object_err+0x1c/0x70 mm/kasan/report.c:166
 print_address_description mm/kasan/report.c:204 [inline]
 kasan_report_error mm/kasan/report.c:288 [inline]
 kasan_report.part.2+0x198/0x440 mm/kasan/report.c:310
 kasan_report mm/kasan/report.c:330 [inline]
 __asan_report_load4_noabort+0x29/0x30 mm/kasan/report.c:330
 rt6_dump_route+0x293/0x2f0 net/ipv6/route.c:3547
 fib6_dump_node+0x101/0x1a0 net/ipv6/ip6_fib.c:315
 fib6_walk_continue+0x4b3/0x620 net/ipv6/ip6_fib.c:1576
 fib6_walk+0x1cf/0x300 net/ipv6/ip6_fib.c:1621
 fib6_dump_table net/ipv6/ip6_fib.c:374 [inline]
 inet6_dump_fib+0x832/0xea0 net/ipv6/ip6_fib.c:447
 rtnl_dump_all+0x8a/0x2a0 net/core/rtnetlink.c:2776
 netlink_dump+0x54d/0xd40 net/netlink/af_netlink.c:2127
 __netlink_dump_start+0x4e5/0x760 net/netlink/af_netlink.c:2217
 netlink_dump_start include/linux/netlink.h:165 [inline]
 rtnetlink_rcv_msg+0x4a3/0x860 net/core/rtnetlink.c:4094
 netlink_rcv_skb+0x2ab/0x390 net/netlink/af_netlink.c:2298
 rtnetlink_rcv+0x2a/0x40 net/core/rtnetlink.c:4110
 netlink_unicast_kernel net/netlink/af_netlink.c:1231 [inline]
 netlink_unicast+0x514/0x730 net/netlink/af_netlink.c:1257
 netlink_sendmsg+0xa9f/0xe50 net/netlink/af_netlink.c:1803
 sock_sendmsg_nosec net/socket.c:633 [inline]
 sock_sendmsg+0xca/0x110 net/socket.c:643
 sock_write_iter+0x326/0x600 net/socket.c:846
 new_sync_write fs/read_write.c:499 [inline]
 __vfs_write+0x483/0x740 fs/read_write.c:512
 vfs_write+0x187/0x530 fs/read_write.c:560
 SYSC_write fs/read_write.c:607 [inline]
 SyS_write+0xfb/0x230 fs/read_write.c:599
 entry_SYSCALL_64_fastpath+0x1f/0xc2
RIP: 0033:0x4458d9
RSP: 002b:00007fe10102bb58 EFLAGS: 00000292 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 0000000000000006 RCX: 00000000004458d9
RDX: 000000000000001f RSI: 0000000020691000 RDI: 0000000000000006
RBP: 00000000006e2fc0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000292 R12: 0000000000708000
R13: 00000000209e1ff7 R14: 0000000000000001 R15: fffffffffffffffd
Object at ffff88004b864400, in cache ip_dst_cache size: 216
Allocated:
PID = 21976
 save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:57
 save_stack+0x43/0xd0 mm/kasan/kasan.c:502
 set_track mm/kasan/kasan.c:514 [inline]
 kasan_kmalloc+0xaa/0xd0 mm/kasan/kasan.c:605
 kasan_slab_alloc+0x12/0x20 mm/kasan/kasan.c:544
 kmem_cache_alloc+0x102/0x680 mm/slab.c:3571
 dst_alloc+0x11b/0x1a0 net/core/dst.c:209
 rt_dst_alloc+0xf0/0x580 net/ipv4/route.c:1482
 __mkroute_output net/ipv4/route.c:2163 [inline]
 __ip_route_output_key_hash+0xce3/0x2c70 net/ipv4/route.c:2373
 __ip_route_output_key include/net/route.h:122 [inline]
 ip_route_output_flow+0x29/0xa0 net/ipv4/route.c:2459
 ip_route_output_key include/net/route.h:132 [inline]
 sctp_v4_get_dst+0x5d2/0x1570 net/sctp/protocol.c:454
 sctp_transport_route+0xa8/0x420 net/sctp/transport.c:292
 sctp_assoc_add_peer+0x5a5/0x1470 net/sctp/associola.c:653
 sctp_sendmsg+0x1800/0x3970 net/sctp/socket.c:1870
 inet_sendmsg+0x164/0x5b0 net/ipv4/af_inet.c:761
 sock_sendmsg_nosec net/socket.c:633 [inline]
 sock_sendmsg+0xca/0x110 net/socket.c:643
 ___sys_sendmsg+0x4a3/0x9f0 net/socket.c:1985
 __sys_sendmmsg+0x25c/0x750 net/socket.c:2075
 SYSC_sendmmsg net/socket.c:2106 [inline]
 SyS_sendmmsg+0x35/0x60 net/socket.c:2101
 entry_SYSCALL_64_fastpath+0x1f/0xc2
Freed:
PID = 15058
 save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:57
 save_stack+0x43/0xd0 mm/kasan/kasan.c:502
 set_track mm/kasan/kasan.c:514 [inline]
 kasan_slab_free+0x6f/0xb0 mm/kasan/kasan.c:578
 __cache_free mm/slab.c:3513 [inline]
 kmem_cache_free+0x71/0x240 mm/slab.c:3773
 dst_destroy+0x1fd/0x330 net/core/dst.c:269
 dst_free include/net/dst.h:428 [inline]
 rt_fibinfo_free_cpus net/ipv4/fib_semantics.c:198 [inline]
 free_fib_info_rcu+0x399/0x590 net/ipv4/fib_semantics.c:213
 __rcu_reclaim kernel/rcu/rcu.h:118 [inline]
 rcu_do_batch.isra.67+0xa31/0xe50 kernel/rcu/tree.c:2877
 invoke_rcu_callbacks kernel/rcu/tree.c:3140 [inline]
 __rcu_process_callbacks kernel/rcu/tree.c:3107 [inline]
 rcu_process_callbacks+0x45b/0xc50 kernel/rcu/tree.c:3124
 __do_softirq+0x31f/0xbe7 kernel/softirq.c:284
Memory state around the buggy address:
 ffff88004b864400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 ffff88004b864480: 00 00 00 00 00 00 00 00 00 00 00 fc fc fc fc fc
>ffff88004b864500: fc fc fc fc fc fc fc fc 00 00 00 00 00 00 00 00
                         ^
 ffff88004b864580: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 ffff88004b864600: 00 00 00 fc fc fc fc fc fc fc fc fc fc fc fc fc
==================================================================

==================================================================
BUG: KASAN: slab-out-of-bounds in fib6_age+0x3fd/0x480
net/ipv6/ip6_fib.c:1769 at addr ffff880088d1bb54
Read of size 4 by task swapper/1/0
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.10.0+ #260
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
 <IRQ>
 __dump_stack lib/dump_stack.c:15 [inline]
 dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
 kasan_object_err+0x1c/0x70 mm/kasan/report.c:166
 print_address_description mm/kasan/report.c:204 [inline]
 kasan_report_error mm/kasan/report.c:288 [inline]
 kasan_report.part.2+0x198/0x440 mm/kasan/report.c:310
 kasan_report mm/kasan/report.c:330 [inline]
 __asan_report_load4_noabort+0x29/0x30 mm/kasan/report.c:330
 fib6_age+0x3fd/0x480 net/ipv6/ip6_fib.c:1769
 fib6_clean_node+0x356/0x550 net/ipv6/ip6_fib.c:1647
 fib6_walk_continue+0x4b3/0x620 net/ipv6/ip6_fib.c:1576
 fib6_walk+0x1cf/0x300 net/ipv6/ip6_fib.c:1621
 fib6_clean_tree+0x266/0x3a0 net/ipv6/ip6_fib.c:1693
 __fib6_clean_all+0x1e1/0x360 net/ipv6/ip6_fib.c:1709
 fib6_clean_all net/ipv6/ip6_fib.c:1720 [inline]
 fib6_run_gc+0x185/0x3d0 net/ipv6/ip6_fib.c:1817
 fib6_gc_timer_cb+0x1c/0x20 net/ipv6/ip6_fib.c:1832
 call_timer_fn+0x241/0x820 kernel/time/timer.c:1266
 expire_timers kernel/time/timer.c:1305 [inline]
 __run_timers+0x960/0xcf0 kernel/time/timer.c:1599
 run_timer_softirq+0x21/0x80 kernel/time/timer.c:1612
 __do_softirq+0x31f/0xbe7 kernel/softirq.c:284
 invoke_softirq kernel/softirq.c:364 [inline]
 irq_exit+0x1cc/0x200 kernel/softirq.c:405
 exiting_irq arch/x86/include/asm/apic.h:658 [inline]
 smp_apic_timer_interrupt+0x76/0xa0 arch/x86/kernel/apic/apic.c:962
 apic_timer_interrupt+0x93/0xa0 arch/x86/entry/entry_64.S:487
RIP: 0010:native_safe_halt+0x6/0x10 arch/x86/include/asm/irqflags.h:53
RSP: 0018:ffff88004dd8fc10 EFLAGS: 00000282 ORIG_RAX: ffffffffffffff10
RAX: dffffc0000000000 RBX: 1ffff10009bb1f85 RCX: 0000000000000000
RDX: 1ffffffff0a18ebc RSI: 0000000000000001 RDI: ffffffff850c75e0
RBP: ffff88004dd8fc10 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 1ffff10009bb1fa9
R13: ffff88004dd8fcc8 R14: ffffffff85697338 R15: ffff88004dd8fe68
 </IRQ>
 arch_safe_halt arch/x86/include/asm/paravirt.h:98 [inline]
 default_idle+0xbf/0x440 arch/x86/kernel/process.c:271
 arch_cpu_idle+0xa/0x10 arch/x86/kernel/process.c:262
 default_idle_call+0x36/0x90 kernel/sched/idle.c:96
 cpuidle_idle_call kernel/sched/idle.c:154 [inline]
 do_idle+0x373/0x520 kernel/sched/idle.c:243
 cpu_startup_entry+0x18/0x20 kernel/sched/idle.c:345
 start_secondary+0x36c/0x460 arch/x86/kernel/smpboot.c:272
 start_cpu+0x14/0x14 arch/x86/kernel/head_64.S:306
Object at ffff880088d1ba40, in cache ip_dst_cache size: 216
Allocated:
PID = 30165
 save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:57
 save_stack+0x43/0xd0 mm/kasan/kasan.c:502
 set_track mm/kasan/kasan.c:514 [inline]
 kasan_kmalloc+0xaa/0xd0 mm/kasan/kasan.c:605
 kasan_slab_alloc+0x12/0x20 mm/kasan/kasan.c:544
 kmem_cache_alloc+0x102/0x680 mm/slab.c:3571
 dst_alloc+0x11b/0x1a0 net/core/dst.c:209
 rt_dst_alloc+0xf0/0x580 net/ipv4/route.c:1482
 __mkroute_output net/ipv4/route.c:2165 [inline]
 __ip_route_output_key_hash+0xce3/0x2c70 net/ipv4/route.c:2375
 __ip_route_output_key include/net/route.h:122 [inline]
 ip_route_output_flow+0x29/0xa0 net/ipv4/route.c:2461
 ip_route_output_key include/net/route.h:132 [inline]
 sctp_v4_get_dst+0x5d2/0x1570 net/sctp/protocol.c:458
 sctp_transport_route+0xa8/0x420 net/sctp/transport.c:292
 sctp_assoc_add_peer+0x5a5/0x1470 net/sctp/associola.c:653
 sctp_sendmsg+0x1800/0x3970 net/sctp/socket.c:1870
 inet_sendmsg+0x164/0x5b0 net/ipv4/af_inet.c:761
 sock_sendmsg_nosec net/socket.c:633 [inline]
 sock_sendmsg+0xca/0x110 net/socket.c:643
 SYSC_sendto+0x660/0x810 net/socket.c:1685
 SyS_sendto+0x40/0x50 net/socket.c:1653
 entry_SYSCALL_64_fastpath+0x1f/0xc2
Freed:
PID = 28880
 save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:57
 save_stack+0x43/0xd0 mm/kasan/kasan.c:502
 set_track mm/kasan/kasan.c:514 [inline]
 kasan_slab_free+0x6f/0xb0 mm/kasan/kasan.c:578
 __cache_free mm/slab.c:3513 [inline]
 kmem_cache_free+0x71/0x240 mm/slab.c:3773
 dst_destroy+0x1fd/0x330 net/core/dst.c:269
 dst_destroy_rcu+0x15/0x40 net/core/dst.c:294
 __rcu_reclaim kernel/rcu/rcu.h:118 [inline]
 rcu_do_batch.isra.67+0xa31/0xe50 kernel/rcu/tree.c:2877
 invoke_rcu_callbacks kernel/rcu/tree.c:3140 [inline]
 __rcu_process_callbacks kernel/rcu/tree.c:3107 [inline]
 rcu_process_callbacks+0x45b/0xc50 kernel/rcu/tree.c:3124
 __do_softirq+0x31f/0xbe7 kernel/softirq.c:284
Memory state around the buggy address:
 ffff880088d1ba00: fc fc fc fc fc fc fc fc 00 00 00 00 00 00 00 00
 ffff880088d1ba80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>ffff880088d1bb00: 00 00 00 fc fc fc fc fc fc fc fc fc fc fc fc fc
                                                 ^
 ffff880088d1bb80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 ffff880088d1bc00: 00 00 00 00 00 00 00 00 00 00 00 fc fc fc fc fc
==================================================================

==================================================================
BUG: KASAN: slab-out-of-bounds in rt6_fill_node.isra.61+0x1434/0x1780
net/ipv6/route.c:3396 at addr ffff88004b5c0790
Read of size 4 by task syz-executor3/3502
CPU: 0 PID: 3502 Comm: syz-executor3 Not tainted 4.10.0+ #260
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:15 [inline]
 dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
 kasan_object_err+0x1c/0x70 mm/kasan/report.c:166
 print_address_description mm/kasan/report.c:204 [inline]
 kasan_report_error mm/kasan/report.c:288 [inline]
 kasan_report.part.2+0x198/0x440 mm/kasan/report.c:310
 kasan_report mm/kasan/report.c:330 [inline]
 __asan_report_load4_noabort+0x29/0x30 mm/kasan/report.c:330
 rt6_fill_node.isra.61+0x1434/0x1780 net/ipv6/route.c:3396
 rt6_dump_route+0x245/0x2f0 net/ipv6/route.c:3557
 fib6_dump_node+0x101/0x1a0 net/ipv6/ip6_fib.c:315
 fib6_walk_continue+0x4b3/0x620 net/ipv6/ip6_fib.c:1576
 fib6_walk+0x1cf/0x300 net/ipv6/ip6_fib.c:1621
 fib6_dump_table net/ipv6/ip6_fib.c:374 [inline]
 inet6_dump_fib+0x832/0xea0 net/ipv6/ip6_fib.c:447
 rtnl_dump_all+0x8a/0x2a0 net/core/rtnetlink.c:2776
 netlink_dump+0x54d/0xd40 net/netlink/af_netlink.c:2127
 netlink_recvmsg+0xb6a/0x1500 net/netlink/af_netlink.c:1886
 sock_recvmsg_nosec net/socket.c:740 [inline]
 sock_recvmsg+0xd7/0x110 net/socket.c:747
 ___sys_recvmsg+0x2b8/0x6b0 net/socket.c:2144
 __sys_recvmsg+0x135/0x300 net/socket.c:2189
 SYSC_recvmsg net/socket.c:2201 [inline]
 SyS_recvmsg+0x2d/0x50 net/socket.c:2196
 do_syscall_64+0x2e8/0x930 arch/x86/entry/common.c:280
 entry_SYSCALL64_slow_path+0x25/0x25
RIP: 0033:0x4458d9
RSP: 002b:00007f694bf1fb58 EFLAGS: 00000286 ORIG_RAX: 000000000000002f
RAX: ffffffffffffffda RBX: 0000000000708000 RCX: 00000000004458d9
RDX: 0000000000000000 RSI: 00000000206a2fc8 RDI: 0000000000000019
RBP: 00000000000036d0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000286 R12: 00000000006e1790
R13: 0000000000000019 R14: 00000000206a2fc8 R15: 0000000000000000
Object at ffff88004b5c0680, in cache ip_dst_cache size: 216
Allocated:
PID = 1362
 save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:57
 save_stack+0x43/0xd0 mm/kasan/kasan.c:502
 set_track mm/kasan/kasan.c:514 [inline]
 kasan_kmalloc+0xaa/0xd0 mm/kasan/kasan.c:605
 kasan_slab_alloc+0x12/0x20 mm/kasan/kasan.c:544
 kmem_cache_alloc+0x102/0x680 mm/slab.c:3571
 dst_alloc+0x11b/0x1a0 net/core/dst.c:209
 rt_dst_alloc+0xf0/0x580 net/ipv4/route.c:1482
 ip_route_input_slow+0xe67/0x21e0 net/ipv4/route.c:1936
 ip_route_input_noref+0x13c/0x10b0 net/ipv4/route.c:2058
 ip_rcv_finish+0x301/0x1b40 net/ipv4/ip_input.c:344
 NF_HOOK include/linux/netfilter.h:257 [inline]
 ip_rcv+0xd75/0x19a0 net/ipv4/ip_input.c:487
 __netif_receive_skb_core+0x1ac8/0x33f0 net/core/dev.c:4179
 __netif_receive_skb+0x2a/0x170 net/core/dev.c:4217
 netif_receive_skb_internal+0xf0/0x400 net/core/dev.c:4245
 napi_skb_finish net/core/dev.c:4602 [inline]
 napi_gro_receive+0x4d4/0x670 net/core/dev.c:4636
 e1000_receive_skb drivers/net/ethernet/intel/e1000/e1000_main.c:4033 [inline]
 e1000_clean_rx_irq+0x5e0/0x1490
drivers/net/ethernet/intel/e1000/e1000_main.c:4489
 e1000_clean+0xb94/0x2920 drivers/net/ethernet/intel/e1000/e1000_main.c:3834
 napi_poll net/core/dev.c:5171 [inline]
 net_rx_action+0xeb4/0x1580 net/core/dev.c:5236
 __do_softirq+0x31f/0xbe7 kernel/softirq.c:284
Freed:
PID = 25328
 save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:57
 save_stack+0x43/0xd0 mm/kasan/kasan.c:502
 set_track mm/kasan/kasan.c:514 [inline]
 kasan_slab_free+0x6f/0xb0 mm/kasan/kasan.c:578
 __cache_free mm/slab.c:3513 [inline]
 kmem_cache_free+0x71/0x240 mm/slab.c:3773
 dst_destroy+0x1fd/0x330 net/core/dst.c:269
 dst_free include/net/dst.h:428 [inline]
 dst_rcu_free+0x152/0x190 include/net/dst.h:438
 __rcu_reclaim kernel/rcu/rcu.h:118 [inline]
 rcu_do_batch.isra.67+0xa31/0xe50 kernel/rcu/tree.c:2877
 invoke_rcu_callbacks kernel/rcu/tree.c:3140 [inline]
 __rcu_process_callbacks kernel/rcu/tree.c:3107 [inline]
 rcu_process_callbacks+0x45b/0xc50 kernel/rcu/tree.c:3124
 __do_softirq+0x31f/0xbe7 kernel/softirq.c:284
Memory state around the buggy address:
 ffff88004b5c0680: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 ffff88004b5c0700: 00 00 00 00 00 00 00 00 00 00 00 fc fc fc fc fc
>ffff88004b5c0780: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb
                         ^
 ffff88004b5c0800: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
 ffff88004b5c0880: fb fb fb fc fc fc fc fc fc fc fc fc fc fc fc fc
==================================================================

==================================================================
BUG: KASAN: slab-out-of-bounds in fib6_prune_clone+0x4e/0x50
net/ipv6/ip6_fib.c:1725 at addr ffff880053497d14
Read of size 4 by task syz-executor1/20792
CPU: 0 PID: 20792 Comm: syz-executor1 Not tainted 4.10.0+ #260
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:15 [inline]
 dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
 kasan_object_err+0x1c/0x70 mm/kasan/report.c:166
 print_address_description mm/kasan/report.c:204 [inline]
 kasan_report_error mm/kasan/report.c:288 [inline]
 kasan_report.part.2+0x198/0x440 mm/kasan/report.c:310
 kasan_report mm/kasan/report.c:330 [inline]
 __asan_report_load4_noabort+0x29/0x30 mm/kasan/report.c:330
 fib6_prune_clone+0x4e/0x50 net/ipv6/ip6_fib.c:1725
 fib6_clean_node+0x356/0x550 net/ipv6/ip6_fib.c:1647
 fib6_walk_continue+0x4b3/0x620 net/ipv6/ip6_fib.c:1576
 fib6_walk+0x1cf/0x300 net/ipv6/ip6_fib.c:1621
 fib6_clean_tree+0x266/0x3a0 net/ipv6/ip6_fib.c:1693
 fib6_prune_clones net/ipv6/ip6_fib.c:1735 [inline]
 fib6_add+0x2612/0x30a0 net/ipv6/ip6_fib.c:1068
 __ip6_ins_rt+0x60/0x80 net/ipv6/route.c:948
 ip6_route_add+0x1a7/0x310 net/ipv6/route.c:2127
 addrconf_prefix_route+0x391/0x560 net/ipv6/addrconf.c:2247
 inet6_addr_add+0x2aa/0x370 net/ipv6/addrconf.c:2799
 addrconf_add_ifaddr+0x169/0x200 net/ipv6/addrconf.c:2878
 inet6_ioctl+0x111/0x1e0 net/ipv6/af_inet6.c:523
 sock_do_ioctl+0x65/0xb0 net/socket.c:895
 sock_ioctl+0x2c2/0x440 net/socket.c:993
 vfs_ioctl fs/ioctl.c:43 [inline]
 do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:683
 SYSC_ioctl fs/ioctl.c:698 [inline]
 SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
 entry_SYSCALL_64_fastpath+0x1f/0xc2
RIP: 0033:0x4458d9
RSP: 002b:00007fce75526b58 EFLAGS: 00000286 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 00000000004458d9
RDX: 0000000020000000 RSI: 0000000000008916 RDI: 0000000000000005
RBP: 00000000006df0c0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000286 R12: 0000000000708000
R13: 0000000020df4ff5 R14: 0000000000000007 R15: 0000000000034800
Object at ffff880053497c00, in cache ip_dst_cache size: 216
Allocated:
PID = 1306
 save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:57
 save_stack+0x43/0xd0 mm/kasan/kasan.c:502
 set_track mm/kasan/kasan.c:514 [inline]
 kasan_kmalloc+0xaa/0xd0 mm/kasan/kasan.c:605
 kasan_slab_alloc+0x12/0x20 mm/kasan/kasan.c:544
 kmem_cache_alloc+0x102/0x680 mm/slab.c:3571
 dst_alloc+0x11b/0x1a0 net/core/dst.c:209
 rt_dst_alloc+0xf0/0x580 net/ipv4/route.c:1482
 __mkroute_output net/ipv4/route.c:2165 [inline]
 __ip_route_output_key_hash+0xce3/0x2c70 net/ipv4/route.c:2375
 __ip_route_output_key include/net/route.h:122 [inline]
 ip_route_output_flow+0x29/0xa0 net/ipv4/route.c:2461
 ip_route_output_ports include/net/route.h:159 [inline]
 ip_queue_xmit+0x1581/0x1a20 net/ipv4/ip_output.c:459
 tcp_transmit_skb+0x1ab4/0x3460 net/ipv4/tcp_output.c:1057
 tcp_write_xmit+0x6e6/0x50d0 net/ipv4/tcp_output.c:2260
 __tcp_push_pending_frames+0xfa/0x380 net/ipv4/tcp_output.c:2445
 tcp_push+0x4e8/0x770 net/ipv4/tcp.c:683
 tcp_sendmsg+0x1275/0x39a0 net/ipv4/tcp.c:1337
 inet_sendmsg+0x164/0x5b0 net/ipv4/af_inet.c:761
 sock_sendmsg_nosec net/socket.c:633 [inline]
 sock_sendmsg+0xca/0x110 net/socket.c:643
 sock_write_iter+0x326/0x600 net/socket.c:846
 new_sync_write fs/read_write.c:499 [inline]
 __vfs_write+0x483/0x740 fs/read_write.c:512
 vfs_write+0x187/0x530 fs/read_write.c:560
 SYSC_write fs/read_write.c:607 [inline]
 SyS_write+0xfb/0x230 fs/read_write.c:599
 entry_SYSCALL_64_fastpath+0x1f/0xc2
Freed:
PID = 0
 save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:57
 save_stack+0x43/0xd0 mm/kasan/kasan.c:502
 set_track mm/kasan/kasan.c:514 [inline]
 kasan_slab_free+0x6f/0xb0 mm/kasan/kasan.c:578
 __cache_free mm/slab.c:3513 [inline]
 kmem_cache_free+0x71/0x240 mm/slab.c:3773
 dst_destroy+0x1fd/0x330 net/core/dst.c:269
 dst_free include/net/dst.h:428 [inline]
 dst_rcu_free+0x152/0x190 include/net/dst.h:438
 __rcu_reclaim kernel/rcu/rcu.h:118 [inline]
 rcu_do_batch.isra.67+0xa31/0xe50 kernel/rcu/tree.c:2877
 invoke_rcu_callbacks kernel/rcu/tree.c:3140 [inline]
 __rcu_process_callbacks kernel/rcu/tree.c:3107 [inline]
 rcu_process_callbacks+0x45b/0xc50 kernel/rcu/tree.c:3124
 __do_softirq+0x31f/0xbe7 kernel/softirq.c:284
Memory state around the buggy address:
 ffff880053497c00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 ffff880053497c80: 00 00 00 00 00 00 00 00 00 00 00 fc fc fc fc fc
>ffff880053497d00: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb
                         ^
 ffff880053497d80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
 ffff880053497e00: fb fb fb fc fc fc fc fc fc fc fc fc fc fc fc fc
==================================================================

==================================================================
BUG: KASAN: slab-out-of-bounds in rt6_fill_node.isra.61+0x1434/0x1780
net/ipv6/route.c:3396 at addr ffff88004af7a650
Read of size 4 by task syz-executor0/14836
CPU: 1 PID: 14836 Comm: syz-executor0 Not tainted 4.10.0+ #260
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:15 [inline]
 dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
9pnet_virtio: no channels available for device ./bus
 kasan_object_err+0x1c/0x70 mm/kasan/report.c:166
 print_address_description mm/kasan/report.c:204 [inline]
 kasan_report_error mm/kasan/report.c:288 [inline]
 kasan_report.part.2+0x198/0x440 mm/kasan/report.c:310
 kasan_report mm/kasan/report.c:330 [inline]
 __asan_report_load4_noabort+0x29/0x30 mm/kasan/report.c:330
 rt6_fill_node.isra.61+0x1434/0x1780 net/ipv6/route.c:3396
 rt6_dump_route+0x245/0x2f0 net/ipv6/route.c:3557
 fib6_dump_node+0x101/0x1a0 net/ipv6/ip6_fib.c:315
 fib6_walk_continue+0x4b3/0x620 net/ipv6/ip6_fib.c:1576
 fib6_walk+0x1cf/0x300 net/ipv6/ip6_fib.c:1621
 fib6_dump_table net/ipv6/ip6_fib.c:374 [inline]
 inet6_dump_fib+0x832/0xea0 net/ipv6/ip6_fib.c:447
 rtnl_dump_all+0x8a/0x2a0 net/core/rtnetlink.c:2776
 netlink_dump+0x54d/0xd40 net/netlink/af_netlink.c:2127
 netlink_recvmsg+0xb6a/0x1500 net/netlink/af_netlink.c:1886
 sock_recvmsg_nosec net/socket.c:740 [inline]
 sock_recvmsg+0xd7/0x110 net/socket.c:747
 ___sys_recvmsg+0x2b8/0x6b0 net/socket.c:2144
 __sys_recvmsg+0x135/0x300 net/socket.c:2189
 SYSC_recvmsg net/socket.c:2201 [inline]
 SyS_recvmsg+0x2d/0x50 net/socket.c:2196
 do_syscall_64+0x2e8/0x930 arch/x86/entry/common.c:280
 entry_SYSCALL64_slow_path+0x25/0x25
RIP: 0033:0x4458d9
RSP: 002b:00007f84c4ef1b58 EFLAGS: 00000286 ORIG_RAX: 000000000000002f
RAX: ffffffffffffffda RBX: 00000000007083f0 RCX: 00000000004458d9
RDX: 0000000000000000 RSI: 00000000206a2fc8 RDI: 000000000000001a
RBP: 00000000000036d0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000286 R12: 00000000006e1790
R13: 000000000000001a R14: 00000000206a2fc8 R15: 0000000000000000
Object at ffff88004af7a540, in cache ip_dst_cache size: 216
Allocated:
PID = 1298
 save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:57
 save_stack+0x43/0xd0 mm/kasan/kasan.c:502
 set_track mm/kasan/kasan.c:514 [inline]
 kasan_kmalloc+0xaa/0xd0 mm/kasan/kasan.c:605
 kasan_slab_alloc+0x12/0x20 mm/kasan/kasan.c:544
 kmem_cache_alloc+0x102/0x680 mm/slab.c:3571
 dst_alloc+0x11b/0x1a0 net/core/dst.c:209
 rt_dst_alloc+0xf0/0x580 net/ipv4/route.c:1482
 ip_route_input_slow+0xe67/0x21e0 net/ipv4/route.c:1936
 ip_route_input_noref+0x13c/0x10b0 net/ipv4/route.c:2058
 ip_rcv_finish+0x301/0x1b40 net/ipv4/ip_input.c:344
 NF_HOOK include/linux/netfilter.h:257 [inline]
 ip_rcv+0xd75/0x19a0 net/ipv4/ip_input.c:487
 __netif_receive_skb_core+0x1ac8/0x33f0 net/core/dev.c:4179
 __netif_receive_skb+0x2a/0x170 net/core/dev.c:4217
 netif_receive_skb_internal+0xf0/0x400 net/core/dev.c:4245
 napi_skb_finish net/core/dev.c:4602 [inline]
 napi_gro_receive+0x4d4/0x670 net/core/dev.c:4636
 e1000_receive_skb drivers/net/ethernet/intel/e1000/e1000_main.c:4033 [inline]
 e1000_clean_rx_irq+0x5e0/0x1490
drivers/net/ethernet/intel/e1000/e1000_main.c:4489
 e1000_clean+0xb94/0x2920 drivers/net/ethernet/intel/e1000/e1000_main.c:3834
 napi_poll net/core/dev.c:5171 [inline]
 net_rx_action+0xeb4/0x1580 net/core/dev.c:5236
 __do_softirq+0x31f/0xbe7 kernel/softirq.c:284
Freed:
PID = 3947
 save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:57
 save_stack+0x43/0xd0 mm/kasan/kasan.c:502
 set_track mm/kasan/kasan.c:514 [inline]
 kasan_slab_free+0x6f/0xb0 mm/kasan/kasan.c:578
 __cache_free mm/slab.c:3513 [inline]
 kmem_cache_free+0x71/0x240 mm/slab.c:3773
 dst_destroy+0x1fd/0x330 net/core/dst.c:269
 dst_destroy_rcu+0x15/0x40 net/core/dst.c:294
 __rcu_reclaim kernel/rcu/rcu.h:118 [inline]
 rcu_do_batch.isra.67+0xa31/0xe50 kernel/rcu/tree.c:2877
 invoke_rcu_callbacks kernel/rcu/tree.c:3140 [inline]
 __rcu_process_callbacks kernel/rcu/tree.c:3107 [inline]
 rcu_process_callbacks+0x45b/0xc50 kernel/rcu/tree.c:3124
 __do_softirq+0x31f/0xbe7 kernel/softirq.c:284
Memory state around the buggy address:
 ffff88004af7a500: fc fc fc fc fc fc fc fc 00 00 00 00 00 00 00 00
 ffff88004af7a580: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>ffff88004af7a600: 00 00 00 fc fc fc fc fc fc fc fc fc fc fc fc fc
                                                 ^
 ffff88004af7a680: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
 ffff88004af7a700: fb fb fb fb fb fb fb fb fb fb fb fc fc fc fc fc
==================================================================

[toc] | [next] | [standalone]

#1592230

On Fri, Mar 3, 2017 at 8:12 PM, David Ahern <dsa@cumulusnetworks.com> wrote:
> On 3/3/17 6:39 AM, Dmitry Vyukov wrote:
>> I am getting heap out-of-bounds reports in
>> fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone while running
>> syzkaller fuzzer on 86292b33d4b79ee03e2f43ea0381ef85f077c760. They all
>> follow the same pattern: an object of size 216 is allocated from
>> ip_dst_cache slab, and then accessed at offset 272/276 withing
>> fib6_walk. Looks like type confusion. Unfortunately this is not
>> reproducible.
>
> I'll take a look this weekend or Monday at the latest.


This is not from fib6_walk, but looks like the same problem:

==================================================================
BUG: KASAN: slab-out-of-bounds in find_rr_leaf net/ipv6/route.c:722
[inline] at addr ffff88004afe6f68
BUG: KASAN: slab-out-of-bounds in rt6_select net/ipv6/route.c:758
[inline] at addr ffff88004afe6f68
BUG: KASAN: slab-out-of-bounds in ip6_pol_route+0x19ff/0x1f30
net/ipv6/route.c:1091 at addr ffff88004afe6f68
Read of size 4 by task syz-executor0/24839
CPU: 1 PID: 24839 Comm: syz-executor0 Not tainted 4.10.0+ #248
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:15 [inline]
 dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
 kasan_object_err+0x1c/0x70 mm/kasan/report.c:166
 print_address_description mm/kasan/report.c:204 [inline]
 kasan_report_error mm/kasan/report.c:288 [inline]
 kasan_report.part.2+0x198/0x440 mm/kasan/report.c:310
 kasan_report mm/kasan/report.c:330 [inline]
 __asan_report_load4_noabort+0x29/0x30 mm/kasan/report.c:330
 find_rr_leaf net/ipv6/route.c:722 [inline]
 rt6_select net/ipv6/route.c:758 [inline]
 ip6_pol_route+0x19ff/0x1f30 net/ipv6/route.c:1091
 ip6_pol_route_output+0x4c/0x60 net/ipv6/route.c:1212
 fib6_rule_lookup+0x52/0x150 net/ipv6/ip6_fib.c:291
 ip6_route_output_flags+0x1f1/0x2b0 net/ipv6/route.c:1240
 ip6_route_output include/net/ip6_route.h:79 [inline]
 ip6_dst_lookup_tail+0x4fb/0x990 net/ipv6/ip6_output.c:954
 ip6_dst_lookup+0x4b/0x60 net/ipv6/ip6_output.c:1056
 icmpv6_route_lookup+0x107/0x750 net/ipv6/icmp.c:347
 icmp6_send+0x145e/0x24d0 net/ipv6/icmp.c:536
 icmpv6_send+0x12e/0x260 net/ipv6/ip6_icmp.c:42
 ip6_fragment+0x57f/0x38a0 net/ipv6/ip6_output.c:865
 ip6_finish_output+0x319/0x950 net/ipv6/ip6_output.c:147
 NF_HOOK_COND include/linux/netfilter.h:246 [inline]
 ip6_output+0x1cb/0x8c0 net/ipv6/ip6_output.c:163
 dst_output include/net/dst.h:486 [inline]
 ip6_local_out+0x95/0x170 net/ipv6/output_core.c:172
 ip6_send_skb+0xa1/0x340 net/ipv6/ip6_output.c:1734
 ip6_push_pending_frames+0xb3/0xe0 net/ipv6/ip6_output.c:1754
 rawv6_push_pending_frames net/ipv6/raw.c:613 [inline]
 rawv6_sendmsg+0x2e10/0x3fd0 net/ipv6/raw.c:930
 inet_sendmsg+0x164/0x5b0 net/ipv4/af_inet.c:761
 sock_sendmsg_nosec net/socket.c:633 [inline]
 sock_sendmsg+0xca/0x110 net/socket.c:643
 SYSC_sendto+0x660/0x810 net/socket.c:1685
 SyS_sendto+0x40/0x50 net/socket.c:1653
 entry_SYSCALL_64_fastpath+0x1f/0xc2
RIP: 0033:0x4458d9
RSP: 002b:00007f227bcfab58 EFLAGS: 00000282 ORIG_RAX: 000000000000002c
RAX: ffffffffffffffda RBX: 0000000000000006 RCX: 00000000004458d9
RDX: 0000000000001001 RSI: 0000000020725000 RDI: 0000000000000006
RBP: 00000000006e1bb0 R08: 00000000201ccff8 R09: 0000000000000018
R10: 0040000000004004 R11: 0000000000000282 R12: 0000000000708000
R13: 0000000020001ff7 R14: 0000000000000003 R15: 0000000000060040
Object at ffff88004afe6e00, in cache ip_dst_cache size: 216
Allocated:
PID = 1307
 save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:57
 save_stack+0x43/0xd0 mm/kasan/kasan.c:502
 set_track mm/kasan/kasan.c:514 [inline]
 kasan_kmalloc+0xaa/0xd0 mm/kasan/kasan.c:605
 kasan_slab_alloc+0x12/0x20 mm/kasan/kasan.c:544
 kmem_cache_alloc+0x102/0x680 mm/slab.c:3571
 dst_alloc+0x11b/0x1a0 net/core/dst.c:209
 rt_dst_alloc+0xf0/0x580 net/ipv4/route.c:1482
 ip_route_input_slow+0xdf2/0x2160 net/ipv4/route.c:1935
 ip_route_input_noref+0x137/0x10e0 net/ipv4/route.c:2056
 ip_rcv_finish+0x301/0x1b40 net/ipv4/ip_input.c:344
 NF_HOOK include/linux/netfilter.h:257 [inline]
 ip_rcv+0xd75/0x19a0 net/ipv4/ip_input.c:487
 __netif_receive_skb_core+0x1ac8/0x33f0 net/core/dev.c:4179
 __netif_receive_skb+0x2a/0x170 net/core/dev.c:4217
 netif_receive_skb_internal+0xf0/0x400 net/core/dev.c:4245
 napi_skb_finish net/core/dev.c:4602 [inline]
 napi_gro_receive+0x4d4/0x670 net/core/dev.c:4636
 e1000_receive_skb drivers/net/ethernet/intel/e1000/e1000_main.c:4033 [inline]
 e1000_clean_rx_irq+0x5e0/0x1490
drivers/net/ethernet/intel/e1000/e1000_main.c:4489
 e1000_clean+0xb94/0x2920 drivers/net/ethernet/intel/e1000/e1000_main.c:3834
 napi_poll net/core/dev.c:5171 [inline]
 net_rx_action+0xeb4/0x1580 net/core/dev.c:5236
 __do_softirq+0x31f/0xbe7 kernel/softirq.c:284
Freed:
PID = 22752
 save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:57
 save_stack+0x43/0xd0 mm/kasan/kasan.c:502
 set_track mm/kasan/kasan.c:514 [inline]
 kasan_slab_free+0x6f/0xb0 mm/kasan/kasan.c:578
 __cache_free mm/slab.c:3513 [inline]
 kmem_cache_free+0x71/0x240 mm/slab.c:3773
 dst_destroy+0x1fd/0x330 net/core/dst.c:269
 dst_destroy_rcu+0x15/0x40 net/core/dst.c:294
 __rcu_reclaim kernel/rcu/rcu.h:118 [inline]
 rcu_do_batch.isra.67+0xa31/0xe50 kernel/rcu/tree.c:2877
 invoke_rcu_callbacks kernel/rcu/tree.c:3140 [inline]
 __rcu_process_callbacks kernel/rcu/tree.c:3107 [inline]
 rcu_process_callbacks+0x45b/0xc50 kernel/rcu/tree.c:3124
 __do_softirq+0x31f/0xbe7 kernel/softirq.c:284
Memory state around the buggy address:
 ffff88004afe6e00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 ffff88004afe6e80: 00 00 00 00 00 00 00 00 00 00 00 fc fc fc fc fc
>ffff88004afe6f00: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
                                                          ^
 ffff88004afe6f80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
 ffff88004afe7000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
==================================================================

[toc] | [prev] | [next] | [standalone]

#1592257 — Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone

On 3/3/17 6:39 AM, Dmitry Vyukov wrote:
> I am getting heap out-of-bounds reports in
> fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone while running
> syzkaller fuzzer on 86292b33d4b79ee03e2f43ea0381ef85f077c760. They all
> follow the same pattern: an object of size 216 is allocated from
> ip_dst_cache slab, and then accessed at offset 272/276 withing
> fib6_walk. Looks like type confusion. Unfortunately this is not
> reproducible.

I'll take a look this weekend or Monday at the latest.

[toc] | [prev] | [next] | [standalone]

#1592604

On Fri, Mar 3, 2017 at 8:12 PM, David Ahern <dsa@cumulusnetworks.com> wrote:
> On 3/3/17 6:39 AM, Dmitry Vyukov wrote:
>> I am getting heap out-of-bounds reports in
>> fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone while running
>> syzkaller fuzzer on 86292b33d4b79ee03e2f43ea0381ef85f077c760. They all
>> follow the same pattern: an object of size 216 is allocated from
>> ip_dst_cache slab, and then accessed at offset 272/276 withing
>> fib6_walk. Looks like type confusion. Unfortunately this is not
>> reproducible.
>
> I'll take a look this weekend or Monday at the latest.


I've got some additional useful info on this. I think this is
use-after-free rather than out-of-bounds. I've collected stack where
the route was disposed with call_rcu, see the last "Disposed" stack.
The crash happens when cmpxchg in rt_cache_route replaces an existing
route. And that route seems to have some existing pointers to it
(rt->dst.rt6_next) which fib6_walk uses to get to it after its
deletion.

==================================================================
BUG: KASAN: slab-out-of-bounds in rt6_dump_route+0x293/0x2f0
net/ipv6/route.c:3551 at addr ffff88007e523694
Read of size 4 by task syz-executor3/24426
CPU: 2 PID: 24426 Comm: syz-executor3 Not tainted 4.10.0+ #293
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:16 [inline]
 dump_stack+0x2fb/0x3fd lib/dump_stack.c:52
 kasan_object_err+0x1c/0x90 mm/kasan/report.c:166
 print_address_description mm/kasan/report.c:208 [inline]
 kasan_report_error mm/kasan/report.c:292 [inline]
 kasan_report.part.2+0x1b0/0x460 mm/kasan/report.c:314
 kasan_report mm/kasan/report.c:334 [inline]
 __asan_report_load4_noabort+0x29/0x30 mm/kasan/report.c:334
 rt6_dump_route+0x293/0x2f0 net/ipv6/route.c:3551
 fib6_dump_node+0x101/0x1a0 net/ipv6/ip6_fib.c:315
 fib6_walk_continue+0x4b3/0x620 net/ipv6/ip6_fib.c:1576
 fib6_walk+0x91/0xf0 net/ipv6/ip6_fib.c:1621
 fib6_dump_table net/ipv6/ip6_fib.c:374 [inline]
 inet6_dump_fib+0x832/0xea0 net/ipv6/ip6_fib.c:447
 rtnl_dump_all+0x8a/0x2a0 net/core/rtnetlink.c:2776
 netlink_dump+0x54d/0xd40 net/netlink/af_netlink.c:2127
 __netlink_dump_start+0x50e/0x790 net/netlink/af_netlink.c:2217
 netlink_dump_start include/linux/netlink.h:165 [inline]
 rtnetlink_rcv_msg+0x4a3/0x860 net/core/rtnetlink.c:4094
 netlink_rcv_skb+0x2ab/0x390 net/netlink/af_netlink.c:2298
 rtnetlink_rcv+0x2a/0x40 net/core/rtnetlink.c:4110
 netlink_unicast_kernel net/netlink/af_netlink.c:1231 [inline]
 netlink_unicast+0x525/0x730 net/netlink/af_netlink.c:1257
 netlink_sendmsg+0xab3/0xe70 net/netlink/af_netlink.c:1803
 sock_sendmsg_nosec net/socket.c:633 [inline]
 sock_sendmsg+0xca/0x110 net/socket.c:643
 sock_write_iter+0x326/0x600 net/socket.c:846
 call_write_iter include/linux/fs.h:1733 [inline]
 new_sync_write fs/read_write.c:497 [inline]
 __vfs_write+0x483/0x740 fs/read_write.c:510
 vfs_write+0x187/0x530 fs/read_write.c:558
 SYSC_write fs/read_write.c:605 [inline]
 SyS_write+0xfb/0x230 fs/read_write.c:597
 entry_SYSCALL_64_fastpath+0x1f/0xc2
RIP: 0033:0x4458d9
RSP: 002b:00007feb6f154b58 EFLAGS: 00000292 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 0000000000000006 RCX: 00000000004458d9
RDX: 000000000000001f RSI: 00000000208a8000 RDI: 0000000000000006
RBP: 00000000006e2fc0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000292 R12: 0000000000708000
R13: 0000000000000005 R14: 0000000020078fd0 R15: 0000000000000030
Object at ffff88007e523580, in cache ip_dst_cache size: 216
Allocated:
PID = 21468
 save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:59
 save_stack+0x43/0xd0 mm/kasan/kasan.c:513
 set_track mm/kasan/kasan.c:525 [inline]
 kasan_kmalloc+0xaa/0xd0 mm/kasan/kasan.c:616
 kasan_slab_alloc+0x12/0x20 mm/kasan/kasan.c:555
 kmem_cache_alloc+0x102/0x6e0 mm/slab.c:3572
 dst_alloc+0x11b/0x1a0 net/core/dst.c:209
 rt_dst_alloc+0xf0/0x580 net/ipv4/route.c:1482
 __mkroute_output net/ipv4/route.c:2165 [inline]
 __ip_route_output_key_hash+0xce3/0x2ca0 net/ipv4/route.c:2375
 __ip_route_output_key include/net/route.h:122 [inline]
 ip_route_connect include/net/route.h:289 [inline]
 tcp_v4_connect+0x11f2/0x2070 net/ipv4/tcp_ipv4.c:170
 __inet_stream_connect+0x2d1/0xf90 net/ipv4/af_inet.c:618
 inet_stream_connect+0x58/0xa0 net/ipv4/af_inet.c:682
 SYSC_connect+0x251/0x580 net/socket.c:1577
 SyS_connect+0x24/0x30 net/socket.c:1558
 entry_SYSCALL_64_fastpath+0x1f/0xc2
Freed:
PID = 20
 save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:59
 save_stack+0x43/0xd0 mm/kasan/kasan.c:513
 set_track mm/kasan/kasan.c:525 [inline]
 kasan_slab_free+0x6f/0xb0 mm/kasan/kasan.c:589
 __cache_free mm/slab.c:3514 [inline]
 kmem_cache_free+0x71/0x240 mm/slab.c:3774
 dst_destroy+0x211/0x340 net/core/dst.c:269
 dst_free include/net/dst.h:428 [inline]
 dst_rcu_free+0x152/0x190 include/net/dst.h:438
 __rcu_reclaim kernel/rcu/rcu.h:118 [inline]
 rcu_do_batch.isra.66+0xa31/0xe50 kernel/rcu/tree.c:2880
 invoke_rcu_callbacks kernel/rcu/tree.c:3143 [inline]
 __rcu_process_callbacks kernel/rcu/tree.c:3110 [inline]
 rcu_process_callbacks+0x45b/0xc50 kernel/rcu/tree.c:3127
 __do_softirq+0x31f/0xbe7 kernel/softirq.c:284
Disposed:
PID = 22571
 save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:59
 save_stack+0x43/0xd0 mm/kasan/kasan.c:513
 set_track mm/kasan/kasan.c:525 [inline]
 kasan_set_rcu_track+0xcf/0xf0 mm/kasan/kasan.c:694
 __call_rcu.constprop.77+0x1d6/0x15a0 kernel/rcu/tree.c:3230
 call_rcu_sched+0x12/0x20 kernel/rcu/tree.c:3291
 rt_free net/ipv4/route.c:592 [inline]
 rt_cache_route+0xf5/0x130 net/ipv4/route.c:1365
 rt_set_nexthop.constprop.57+0x408/0xfa0 net/ipv4/route.c:1453
 __mkroute_output net/ipv4/route.c:2195 [inline]
 __ip_route_output_key_hash+0xe50/0x2ca0 net/ipv4/route.c:2375
 __ip_route_output_key include/net/route.h:122 [inline]
 ip_route_output_flow+0x29/0xa0 net/ipv4/route.c:2461
 ip_route_connect include/net/route.h:296 [inline]
 tcp_v4_connect+0x784/0x2070 net/ipv4/tcp_ipv4.c:170
 __inet_stream_connect+0x2d1/0xf90 net/ipv4/af_inet.c:618
 inet_stream_connect+0x58/0xa0 net/ipv4/af_inet.c:682
 SYSC_connect+0x251/0x580 net/socket.c:1577
 SyS_connect+0x24/0x30 net/socket.c:1558
 entry_SYSCALL_64_fastpath+0x1f/0xc2
Memory state around the buggy address:
 ffff88007e523580: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 ffff88007e523600: 00 00 00 00 00 00 00 00 00 00 00 fc fc fc fc fc
>ffff88007e523680: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb
                         ^
 ffff88007e523700: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
 ffff88007e523780: fb fb fb fc fc fc fc fc fc fc fc fc fc fc fc fc
==================================================================

[toc] | [prev] | [next] | [standalone]

#1592605

On Sat, Mar 4, 2017 at 7:57 PM, Dmitry Vyukov <dvyukov@google.com> wrote:
> On Fri, Mar 3, 2017 at 8:12 PM, David Ahern <dsa@cumulusnetworks.com> wrote:
>> On 3/3/17 6:39 AM, Dmitry Vyukov wrote:
>>> I am getting heap out-of-bounds reports in
>>> fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone while running
>>> syzkaller fuzzer on 86292b33d4b79ee03e2f43ea0381ef85f077c760. They all
>>> follow the same pattern: an object of size 216 is allocated from
>>> ip_dst_cache slab, and then accessed at offset 272/276 withing
>>> fib6_walk. Looks like type confusion. Unfortunately this is not
>>> reproducible.
>>
>> I'll take a look this weekend or Monday at the latest.
>
>
> I've got some additional useful info on this. I think this is
> use-after-free rather than out-of-bounds. I've collected stack where
> the route was disposed with call_rcu, see the last "Disposed" stack.
> The crash happens when cmpxchg in rt_cache_route replaces an existing
> route. And that route seems to have some existing pointers to it
> (rt->dst.rt6_next) which fib6_walk uses to get to it after its
> deletion.


This could explain the gazillion of various crashes I am seeing in ip6 routes:

KASAN: use-after-free Read in fib6_add
WARNING in fib6_del
KASAN: use-after-free Read in ip6_pol_route
general protection fault in fib6_add
general protection fault in ip6_rt_cache_alloc
general protection fault in sctp_v6_get_dst
KASAN: slab-out-of-bounds Read in rt6_dump_route
KASAN: slab-out-of-bounds Read in fib6_prune_clone
KASAN: slab-out-of-bounds Read in fib6_age
KASAN: slab-out-of-bounds Read in ip6_pol_route
KASAN: slab-out-of-bounds Read in rt6_fill_node


> ==================================================================
> BUG: KASAN: slab-out-of-bounds in rt6_dump_route+0x293/0x2f0
> net/ipv6/route.c:3551 at addr ffff88007e523694
> Read of size 4 by task syz-executor3/24426
> CPU: 2 PID: 24426 Comm: syz-executor3 Not tainted 4.10.0+ #293
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:16 [inline]
>  dump_stack+0x2fb/0x3fd lib/dump_stack.c:52
>  kasan_object_err+0x1c/0x90 mm/kasan/report.c:166
>  print_address_description mm/kasan/report.c:208 [inline]
>  kasan_report_error mm/kasan/report.c:292 [inline]
>  kasan_report.part.2+0x1b0/0x460 mm/kasan/report.c:314
>  kasan_report mm/kasan/report.c:334 [inline]
>  __asan_report_load4_noabort+0x29/0x30 mm/kasan/report.c:334
>  rt6_dump_route+0x293/0x2f0 net/ipv6/route.c:3551
>  fib6_dump_node+0x101/0x1a0 net/ipv6/ip6_fib.c:315
>  fib6_walk_continue+0x4b3/0x620 net/ipv6/ip6_fib.c:1576
>  fib6_walk+0x91/0xf0 net/ipv6/ip6_fib.c:1621
>  fib6_dump_table net/ipv6/ip6_fib.c:374 [inline]
>  inet6_dump_fib+0x832/0xea0 net/ipv6/ip6_fib.c:447
>  rtnl_dump_all+0x8a/0x2a0 net/core/rtnetlink.c:2776
>  netlink_dump+0x54d/0xd40 net/netlink/af_netlink.c:2127
>  __netlink_dump_start+0x50e/0x790 net/netlink/af_netlink.c:2217
>  netlink_dump_start include/linux/netlink.h:165 [inline]
>  rtnetlink_rcv_msg+0x4a3/0x860 net/core/rtnetlink.c:4094
>  netlink_rcv_skb+0x2ab/0x390 net/netlink/af_netlink.c:2298
>  rtnetlink_rcv+0x2a/0x40 net/core/rtnetlink.c:4110
>  netlink_unicast_kernel net/netlink/af_netlink.c:1231 [inline]
>  netlink_unicast+0x525/0x730 net/netlink/af_netlink.c:1257
>  netlink_sendmsg+0xab3/0xe70 net/netlink/af_netlink.c:1803
>  sock_sendmsg_nosec net/socket.c:633 [inline]
>  sock_sendmsg+0xca/0x110 net/socket.c:643
>  sock_write_iter+0x326/0x600 net/socket.c:846
>  call_write_iter include/linux/fs.h:1733 [inline]
>  new_sync_write fs/read_write.c:497 [inline]
>  __vfs_write+0x483/0x740 fs/read_write.c:510
>  vfs_write+0x187/0x530 fs/read_write.c:558
>  SYSC_write fs/read_write.c:605 [inline]
>  SyS_write+0xfb/0x230 fs/read_write.c:597
>  entry_SYSCALL_64_fastpath+0x1f/0xc2
> RIP: 0033:0x4458d9
> RSP: 002b:00007feb6f154b58 EFLAGS: 00000292 ORIG_RAX: 0000000000000001
> RAX: ffffffffffffffda RBX: 0000000000000006 RCX: 00000000004458d9
> RDX: 000000000000001f RSI: 00000000208a8000 RDI: 0000000000000006
> RBP: 00000000006e2fc0 R08: 0000000000000000 R09: 0000000000000000
> R10: 0000000000000000 R11: 0000000000000292 R12: 0000000000708000
> R13: 0000000000000005 R14: 0000000020078fd0 R15: 0000000000000030
> Object at ffff88007e523580, in cache ip_dst_cache size: 216
> Allocated:
> PID = 21468
>  save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:59
>  save_stack+0x43/0xd0 mm/kasan/kasan.c:513
>  set_track mm/kasan/kasan.c:525 [inline]
>  kasan_kmalloc+0xaa/0xd0 mm/kasan/kasan.c:616
>  kasan_slab_alloc+0x12/0x20 mm/kasan/kasan.c:555
>  kmem_cache_alloc+0x102/0x6e0 mm/slab.c:3572
>  dst_alloc+0x11b/0x1a0 net/core/dst.c:209
>  rt_dst_alloc+0xf0/0x580 net/ipv4/route.c:1482
>  __mkroute_output net/ipv4/route.c:2165 [inline]
>  __ip_route_output_key_hash+0xce3/0x2ca0 net/ipv4/route.c:2375
>  __ip_route_output_key include/net/route.h:122 [inline]
>  ip_route_connect include/net/route.h:289 [inline]
>  tcp_v4_connect+0x11f2/0x2070 net/ipv4/tcp_ipv4.c:170
>  __inet_stream_connect+0x2d1/0xf90 net/ipv4/af_inet.c:618
>  inet_stream_connect+0x58/0xa0 net/ipv4/af_inet.c:682
>  SYSC_connect+0x251/0x580 net/socket.c:1577
>  SyS_connect+0x24/0x30 net/socket.c:1558
>  entry_SYSCALL_64_fastpath+0x1f/0xc2
> Freed:
> PID = 20
>  save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:59
>  save_stack+0x43/0xd0 mm/kasan/kasan.c:513
>  set_track mm/kasan/kasan.c:525 [inline]
>  kasan_slab_free+0x6f/0xb0 mm/kasan/kasan.c:589
>  __cache_free mm/slab.c:3514 [inline]
>  kmem_cache_free+0x71/0x240 mm/slab.c:3774
>  dst_destroy+0x211/0x340 net/core/dst.c:269
>  dst_free include/net/dst.h:428 [inline]
>  dst_rcu_free+0x152/0x190 include/net/dst.h:438
>  __rcu_reclaim kernel/rcu/rcu.h:118 [inline]
>  rcu_do_batch.isra.66+0xa31/0xe50 kernel/rcu/tree.c:2880
>  invoke_rcu_callbacks kernel/rcu/tree.c:3143 [inline]
>  __rcu_process_callbacks kernel/rcu/tree.c:3110 [inline]
>  rcu_process_callbacks+0x45b/0xc50 kernel/rcu/tree.c:3127
>  __do_softirq+0x31f/0xbe7 kernel/softirq.c:284
> Disposed:
> PID = 22571
>  save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:59
>  save_stack+0x43/0xd0 mm/kasan/kasan.c:513
>  set_track mm/kasan/kasan.c:525 [inline]
>  kasan_set_rcu_track+0xcf/0xf0 mm/kasan/kasan.c:694
>  __call_rcu.constprop.77+0x1d6/0x15a0 kernel/rcu/tree.c:3230
>  call_rcu_sched+0x12/0x20 kernel/rcu/tree.c:3291
>  rt_free net/ipv4/route.c:592 [inline]
>  rt_cache_route+0xf5/0x130 net/ipv4/route.c:1365
>  rt_set_nexthop.constprop.57+0x408/0xfa0 net/ipv4/route.c:1453
>  __mkroute_output net/ipv4/route.c:2195 [inline]
>  __ip_route_output_key_hash+0xe50/0x2ca0 net/ipv4/route.c:2375
>  __ip_route_output_key include/net/route.h:122 [inline]
>  ip_route_output_flow+0x29/0xa0 net/ipv4/route.c:2461
>  ip_route_connect include/net/route.h:296 [inline]
>  tcp_v4_connect+0x784/0x2070 net/ipv4/tcp_ipv4.c:170
>  __inet_stream_connect+0x2d1/0xf90 net/ipv4/af_inet.c:618
>  inet_stream_connect+0x58/0xa0 net/ipv4/af_inet.c:682
>  SYSC_connect+0x251/0x580 net/socket.c:1577
>  SyS_connect+0x24/0x30 net/socket.c:1558
>  entry_SYSCALL_64_fastpath+0x1f/0xc2
> Memory state around the buggy address:
>  ffff88007e523580: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>  ffff88007e523600: 00 00 00 00 00 00 00 00 00 00 00 fc fc fc fc fc
>>ffff88007e523680: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb
>                          ^
>  ffff88007e523700: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
>  ffff88007e523780: fb fb fb fc fc fc fc fc fc fc fc fc fc fc fc fc
> ==================================================================

[toc] | [prev] | [next] | [standalone]

#1592614 — Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone

On Sat, 2017-03-04 at 19:57 +0100, Dmitry Vyukov wrote:
> On Fri, Mar 3, 2017 at 8:12 PM, David Ahern <dsa@cumulusnetworks.com> wrote:
> > On 3/3/17 6:39 AM, Dmitry Vyukov wrote:
> >> I am getting heap out-of-bounds reports in
> >> fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone while running
> >> syzkaller fuzzer on 86292b33d4b79ee03e2f43ea0381ef85f077c760. They all
> >> follow the same pattern: an object of size 216 is allocated from
> >> ip_dst_cache slab, and then accessed at offset 272/276 withing
> >> fib6_walk. Looks like type confusion. Unfortunately this is not
> >> reproducible.
> >
> > I'll take a look this weekend or Monday at the latest.
> 
> 
> I've got some additional useful info on this. I think this is
> use-after-free rather than out-of-bounds. I've collected stack where
> the route was disposed with call_rcu, see the last "Disposed" stack.
> The crash happens when cmpxchg in rt_cache_route replaces an existing
> route. And that route seems to have some existing pointers to it
> (rt->dst.rt6_next) which fib6_walk uses to get to it after its
> deletion.

rt_cache_route() deals with IPv4 routes.

We somehow mix IPv4 and IPv6 dsts in IPv6 tree.

We need to add type safety at IPV6 route insertions to catch the
offender.

[toc] | [prev] | [next] | [standalone]

#1592728

On Sat, Mar 4, 2017 at 9:15 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
>> > On 3/3/17 6:39 AM, Dmitry Vyukov wrote:
>> >> I am getting heap out-of-bounds reports in
>> >> fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone while running
>> >> syzkaller fuzzer on 86292b33d4b79ee03e2f43ea0381ef85f077c760. They all
>> >> follow the same pattern: an object of size 216 is allocated from
>> >> ip_dst_cache slab, and then accessed at offset 272/276 withing
>> >> fib6_walk. Looks like type confusion. Unfortunately this is not
>> >> reproducible.
>> >
>> > I'll take a look this weekend or Monday at the latest.
>>
>>
>> I've got some additional useful info on this. I think this is
>> use-after-free rather than out-of-bounds. I've collected stack where
>> the route was disposed with call_rcu, see the last "Disposed" stack.
>> The crash happens when cmpxchg in rt_cache_route replaces an existing
>> route. And that route seems to have some existing pointers to it
>> (rt->dst.rt6_next) which fib6_walk uses to get to it after its
>> deletion.
>
> rt_cache_route() deals with IPv4 routes.
>
> We somehow mix IPv4 and IPv6 dsts in IPv6 tree.
>
> We need to add type safety at IPV6 route insertions to catch the
> offender.


If you suggest additional checks, I will collect stacks.

[toc] | [prev] | [next] | [standalone]

#1593546 — Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone

On 3/4/17 1:15 PM, Eric Dumazet wrote:
> On Sat, 2017-03-04 at 19:57 +0100, Dmitry Vyukov wrote:
>> On Fri, Mar 3, 2017 at 8:12 PM, David Ahern <dsa@cumulusnetworks.com> wrote:
>>> On 3/3/17 6:39 AM, Dmitry Vyukov wrote:
>>>> I am getting heap out-of-bounds reports in
>>>> fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone while running
>>>> syzkaller fuzzer on 86292b33d4b79ee03e2f43ea0381ef85f077c760. They all
>>>> follow the same pattern: an object of size 216 is allocated from
>>>> ip_dst_cache slab, and then accessed at offset 272/276 withing
>>>> fib6_walk. Looks like type confusion. Unfortunately this is not
>>>> reproducible.
>>>
>>> I'll take a look this weekend or Monday at the latest.
>>
>>
>> I've got some additional useful info on this. I think this is
>> use-after-free rather than out-of-bounds. I've collected stack where
>> the route was disposed with call_rcu, see the last "Disposed" stack.
>> The crash happens when cmpxchg in rt_cache_route replaces an existing
>> route. And that route seems to have some existing pointers to it
>> (rt->dst.rt6_next) which fib6_walk uses to get to it after its
>> deletion.
> 
> rt_cache_route() deals with IPv4 routes.
> 
> We somehow mix IPv4 and IPv6 dsts in IPv6 tree.
> 
> We need to add type safety at IPV6 route insertions to catch the
> offender.
> 

I've seen something like this before -- a rt was on the gc list but
still linked in the tables because of some reference.

Dmitry: you seem to have reproduced this a few times. Can you share how
to run whatever tests you are using?

[toc] | [prev] | [next] | [standalone]

#1593595

On Mon, Mar 6, 2017 at 6:31 PM, David Ahern <dsa@cumulusnetworks.com> wrote:
> On 3/4/17 1:15 PM, Eric Dumazet wrote:
>> On Sat, 2017-03-04 at 19:57 +0100, Dmitry Vyukov wrote:
>>> On Fri, Mar 3, 2017 at 8:12 PM, David Ahern <dsa@cumulusnetworks.com> wrote:
>>>> On 3/3/17 6:39 AM, Dmitry Vyukov wrote:
>>>>> I am getting heap out-of-bounds reports in
>>>>> fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone while running
>>>>> syzkaller fuzzer on 86292b33d4b79ee03e2f43ea0381ef85f077c760. They all
>>>>> follow the same pattern: an object of size 216 is allocated from
>>>>> ip_dst_cache slab, and then accessed at offset 272/276 withing
>>>>> fib6_walk. Looks like type confusion. Unfortunately this is not
>>>>> reproducible.
>>>>
>>>> I'll take a look this weekend or Monday at the latest.
>>>
>>>
>>> I've got some additional useful info on this. I think this is
>>> use-after-free rather than out-of-bounds. I've collected stack where
>>> the route was disposed with call_rcu, see the last "Disposed" stack.
>>> The crash happens when cmpxchg in rt_cache_route replaces an existing
>>> route. And that route seems to have some existing pointers to it
>>> (rt->dst.rt6_next) which fib6_walk uses to get to it after its
>>> deletion.
>>
>> rt_cache_route() deals with IPv4 routes.
>>
>> We somehow mix IPv4 and IPv6 dsts in IPv6 tree.
>>
>> We need to add type safety at IPV6 route insertions to catch the
>> offender.
>>
>
> I've seen something like this before -- a rt was on the gc list but
> still linked in the tables because of some reference.
>
> Dmitry: you seem to have reproduced this a few times. Can you share how
> to run whatever tests you are using?


We hit it several thousand times, but we get only several dozens of
crashes per day on ~80 VMs. So if you try to reproduce it on a single
machine it can take days for a single crash.
If you are ready to go that route, here are some instructions on
setting up syzkaller:
https://github.com/google/syzkaller
You also need kernel built with CONFIG_KASAN.
I am ready to help with resolving any issues.

Another possible route is if you give me a patch with some additional
WARNINGs. Then I can deploy it to bots and collect stacks.

[toc] | [prev] | [next] | [standalone]

#1593806 — Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone

[Multipart message — attachments visible in raw view] — view raw

On 3/6/17 11:51 AM, Dmitry Vyukov wrote:
> We hit it several thousand times, but we get only several dozens of
> crashes per day on ~80 VMs. So if you try to reproduce it on a single
> machine it can take days for a single crash.
> If you are ready to go that route, here are some instructions on
> setting up syzkaller:
> https://github.com/google/syzkaller
> You also need kernel built with CONFIG_KASAN.

ack and I have it setup on ubuntu 16.10 which has a fairly new compiler.

> I am ready to help with resolving any issues.
> 
> Another possible route is if you give me a patch with some additional
> WARNINGs. Then I can deploy it to bots and collect stacks.

try the attached.

[toc] | [prev] | [next] | [standalone]

#1593992

On Tue, Mar 7, 2017 at 12:41 AM, David Ahern <dsa@cumulusnetworks.com> wrote:
> On 3/6/17 11:51 AM, Dmitry Vyukov wrote:
>> We hit it several thousand times, but we get only several dozens of
>> crashes per day on ~80 VMs. So if you try to reproduce it on a single
>> machine it can take days for a single crash.
>> If you are ready to go that route, here are some instructions on
>> setting up syzkaller:
>> https://github.com/google/syzkaller
>> You also need kernel built with CONFIG_KASAN.
>
> ack and I have it setup on ubuntu 16.10 which has a fairly new compiler.
>
>> I am ready to help with resolving any issues.
>>
>> Another possible route is if you give me a patch with some additional
>> WARNINGs. Then I can deploy it to bots and collect stacks.
>
> try the attached.


This is on c1ae3cfa0e89fa1a7ecc4c99031f5e9ae99d9201. No other kernel
output from your patch (pr_err).

------------[ cut here ]------------
WARNING: CPU: 1 PID: 30179 at net/ipv6/ip6_fib.c:158
rt6_rcu_free+0x61/0x70 net/ipv6/ip6_fib.c:158
Kernel panic - not syncing: panic_on_warn set ...

CPU: 1 PID: 30179 Comm: syz-executor3 Not tainted 4.11.0-rc1+ #310
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:16 [inline]
 dump_stack+0x2fb/0x3fd lib/dump_stack.c:52
 panic+0x20f/0x426 kernel/panic.c:180
 __warn+0x1c4/0x1e0 kernel/panic.c:541
 warn_slowpath_null+0x2c/0x40 kernel/panic.c:584
 rt6_rcu_free+0x61/0x70 net/ipv6/ip6_fib.c:158
 rt6_release+0x1ee/0x290 net/ipv6/ip6_fib.c:189
 fib6_add_rt2node net/ipv6/ip6_fib.c:922 [inline]
 fib6_add+0x1d51/0x3290 net/ipv6/ip6_fib.c:1081
 __ip6_ins_rt+0x60/0x80 net/ipv6/route.c:948
 ip6_route_add+0x1a7/0x310 net/ipv6/route.c:2130
 inet6_rtm_newroute+0x191/0x1b0 net/ipv6/route.c:3294
 rtnetlink_rcv_msg+0x609/0x860 net/core/rtnetlink.c:4104
 netlink_rcv_skb+0x2ab/0x390 net/netlink/af_netlink.c:2298
 rtnetlink_rcv+0x2a/0x40 net/core/rtnetlink.c:4110
 netlink_unicast_kernel net/netlink/af_netlink.c:1231 [inline]
 netlink_unicast+0x525/0x730 net/netlink/af_netlink.c:1257
 netlink_sendmsg+0xab3/0xe70 net/netlink/af_netlink.c:1803
 sock_sendmsg_nosec net/socket.c:633 [inline]
 sock_sendmsg+0xca/0x110 net/socket.c:643
 sock_write_iter+0x326/0x600 net/socket.c:846
 call_write_iter include/linux/fs.h:1733 [inline]
 do_iter_readv_writev fs/read_write.c:696 [inline]
 __do_readv_writev+0xbbc/0x10a0 fs/read_write.c:862
 do_readv_writev+0x13f/0x200 fs/read_write.c:894
 vfs_writev+0x87/0xc0 fs/read_write.c:921
 do_writev+0x110/0x2c0 fs/read_write.c:954
 SYSC_writev fs/read_write.c:1027 [inline]
 SyS_writev+0x27/0x30 fs/read_write.c:1024
 entry_SYSCALL_64_fastpath+0x1f/0xc2
RIP: 0033:0x4458d9
RSP: 002b:00007f31fcf33b58 EFLAGS: 00000292 ORIG_RAX: 0000000000000014
RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 00000000004458d9
RDX: 0000000000000001 RSI: 00000000207cd000 RDI: 0000000000000005
RBP: 00000000006e30c0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000292 R12: 0000000000708000
R13: 0000000020fad000 R14: 0000000000001000 R15: 0000000000000003



------------[ cut here ]------------
WARNING: CPU: 2 PID: 31175 at net/ipv6/ip6_fib.c:158
rt6_rcu_free+0x61/0x70 net/ipv6/ip6_fib.c:158
Kernel panic - not syncing: panic_on_warn set ...

CPU: 2 PID: 31175 Comm: syz-executor1 Not tainted 4.11.0-rc1+ #310
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:16 [inline]
 dump_stack+0x2fb/0x3fd lib/dump_stack.c:52
 panic+0x20f/0x426 kernel/panic.c:180
 __warn+0x1c4/0x1e0 kernel/panic.c:541
 warn_slowpath_null+0x2c/0x40 kernel/panic.c:584
 rt6_rcu_free+0x61/0x70 net/ipv6/ip6_fib.c:158
 rt6_release+0x1ee/0x290 net/ipv6/ip6_fib.c:189
 fib6_add_rt2node net/ipv6/ip6_fib.c:922 [inline]
 fib6_add+0x1d51/0x3290 net/ipv6/ip6_fib.c:1081
kvm_vm_ioctl_deassign_device: device hasn't been assigned before, so
cannot be deassigned
 __ip6_ins_rt+0x60/0x80 net/ipv6/route.c:948
 ip6_route_add+0x1a7/0x310 net/ipv6/route.c:2130
 inet6_rtm_newroute+0x191/0x1b0 net/ipv6/route.c:3294
 rtnetlink_rcv_msg+0x609/0x860 net/core/rtnetlink.c:4104
 netlink_rcv_skb+0x2ab/0x390 net/netlink/af_netlink.c:2298
 rtnetlink_rcv+0x2a/0x40 net/core/rtnetlink.c:4110
 netlink_unicast_kernel net/netlink/af_netlink.c:1231 [inline]
 netlink_unicast+0x525/0x730 net/netlink/af_netlink.c:1257
 netlink_sendmsg+0xab3/0xe70 net/netlink/af_netlink.c:1803
 sock_sendmsg_nosec net/socket.c:633 [inline]
 sock_sendmsg+0xca/0x110 net/socket.c:643
 sock_write_iter+0x326/0x600 net/socket.c:846
 call_write_iter include/linux/fs.h:1733 [inline]
 do_iter_readv_writev fs/read_write.c:696 [inline]
 __do_readv_writev+0xbbc/0x10a0 fs/read_write.c:862
 do_readv_writev+0x13f/0x200 fs/read_write.c:894
 vfs_writev+0x87/0xc0 fs/read_write.c:921
 do_writev+0x110/0x2c0 fs/read_write.c:954
 SYSC_writev fs/read_write.c:1027 [inline]
 SyS_writev+0x27/0x30 fs/read_write.c:1024
 entry_SYSCALL_64_fastpath+0x1f/0xc2
RIP: 0033:0x4458d9
RSP: 002b:00007f1639006b58 EFLAGS: 00000292 ORIG_RAX: 0000000000000014
RAX: ffffffffffffffda RBX: 0000000000000019 RCX: 00000000004458d9
RDX: 0000000000000001 RSI: 00000000207cd000 RDI: 0000000000000019
RBP: 00000000006e30c0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000292 R12: 0000000000708000
R13: 0000000000000010 R14: 0000000000000003 R15: 0000000000000000

[toc] | [prev] | [next] | [standalone]

#1594089

On Tue, Mar 7, 2017 at 9:43 AM, Dmitry Vyukov <dvyukov@google.com> wrote:
> On Tue, Mar 7, 2017 at 12:41 AM, David Ahern <dsa@cumulusnetworks.com> wrote:
>> On 3/6/17 11:51 AM, Dmitry Vyukov wrote:
>>> We hit it several thousand times, but we get only several dozens of
>>> crashes per day on ~80 VMs. So if you try to reproduce it on a single
>>> machine it can take days for a single crash.
>>> If you are ready to go that route, here are some instructions on
>>> setting up syzkaller:
>>> https://github.com/google/syzkaller
>>> You also need kernel built with CONFIG_KASAN.
>>
>> ack and I have it setup on ubuntu 16.10 which has a fairly new compiler.
>>
>>> I am ready to help with resolving any issues.
>>>
>>> Another possible route is if you give me a patch with some additional
>>> WARNINGs. Then I can deploy it to bots and collect stacks.
>>
>> try the attached.
>
>
> This is on c1ae3cfa0e89fa1a7ecc4c99031f5e9ae99d9201. No other kernel
> output from your patch (pr_err).
>
> ------------[ cut here ]------------
> WARNING: CPU: 1 PID: 30179 at net/ipv6/ip6_fib.c:158
> rt6_rcu_free+0x61/0x70 net/ipv6/ip6_fib.c:158
> Kernel panic - not syncing: panic_on_warn set ...
>
> CPU: 1 PID: 30179 Comm: syz-executor3 Not tainted 4.11.0-rc1+ #310
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:16 [inline]
>  dump_stack+0x2fb/0x3fd lib/dump_stack.c:52
>  panic+0x20f/0x426 kernel/panic.c:180
>  __warn+0x1c4/0x1e0 kernel/panic.c:541
>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:584
>  rt6_rcu_free+0x61/0x70 net/ipv6/ip6_fib.c:158
>  rt6_release+0x1ee/0x290 net/ipv6/ip6_fib.c:189
>  fib6_add_rt2node net/ipv6/ip6_fib.c:922 [inline]
>  fib6_add+0x1d51/0x3290 net/ipv6/ip6_fib.c:1081
>  __ip6_ins_rt+0x60/0x80 net/ipv6/route.c:948
>  ip6_route_add+0x1a7/0x310 net/ipv6/route.c:2130
>  inet6_rtm_newroute+0x191/0x1b0 net/ipv6/route.c:3294
>  rtnetlink_rcv_msg+0x609/0x860 net/core/rtnetlink.c:4104
>  netlink_rcv_skb+0x2ab/0x390 net/netlink/af_netlink.c:2298
>  rtnetlink_rcv+0x2a/0x40 net/core/rtnetlink.c:4110
>  netlink_unicast_kernel net/netlink/af_netlink.c:1231 [inline]
>  netlink_unicast+0x525/0x730 net/netlink/af_netlink.c:1257
>  netlink_sendmsg+0xab3/0xe70 net/netlink/af_netlink.c:1803
>  sock_sendmsg_nosec net/socket.c:633 [inline]
>  sock_sendmsg+0xca/0x110 net/socket.c:643
>  sock_write_iter+0x326/0x600 net/socket.c:846
>  call_write_iter include/linux/fs.h:1733 [inline]
>  do_iter_readv_writev fs/read_write.c:696 [inline]
>  __do_readv_writev+0xbbc/0x10a0 fs/read_write.c:862
>  do_readv_writev+0x13f/0x200 fs/read_write.c:894
>  vfs_writev+0x87/0xc0 fs/read_write.c:921
>  do_writev+0x110/0x2c0 fs/read_write.c:954
>  SYSC_writev fs/read_write.c:1027 [inline]
>  SyS_writev+0x27/0x30 fs/read_write.c:1024
>  entry_SYSCALL_64_fastpath+0x1f/0xc2
> RIP: 0033:0x4458d9
> RSP: 002b:00007f31fcf33b58 EFLAGS: 00000292 ORIG_RAX: 0000000000000014
> RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 00000000004458d9
> RDX: 0000000000000001 RSI: 00000000207cd000 RDI: 0000000000000005
> RBP: 00000000006e30c0 R08: 0000000000000000 R09: 0000000000000000
> R10: 0000000000000000 R11: 0000000000000292 R12: 0000000000708000
> R13: 0000000020fad000 R14: 0000000000001000 R15: 0000000000000003
>
>
>
> ------------[ cut here ]------------
> WARNING: CPU: 2 PID: 31175 at net/ipv6/ip6_fib.c:158
> rt6_rcu_free+0x61/0x70 net/ipv6/ip6_fib.c:158
> Kernel panic - not syncing: panic_on_warn set ...
>
> CPU: 2 PID: 31175 Comm: syz-executor1 Not tainted 4.11.0-rc1+ #310
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:16 [inline]
>  dump_stack+0x2fb/0x3fd lib/dump_stack.c:52
>  panic+0x20f/0x426 kernel/panic.c:180
>  __warn+0x1c4/0x1e0 kernel/panic.c:541
>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:584
>  rt6_rcu_free+0x61/0x70 net/ipv6/ip6_fib.c:158
>  rt6_release+0x1ee/0x290 net/ipv6/ip6_fib.c:189
>  fib6_add_rt2node net/ipv6/ip6_fib.c:922 [inline]
>  fib6_add+0x1d51/0x3290 net/ipv6/ip6_fib.c:1081
> kvm_vm_ioctl_deassign_device: device hasn't been assigned before, so
> cannot be deassigned
>  __ip6_ins_rt+0x60/0x80 net/ipv6/route.c:948
>  ip6_route_add+0x1a7/0x310 net/ipv6/route.c:2130
>  inet6_rtm_newroute+0x191/0x1b0 net/ipv6/route.c:3294
>  rtnetlink_rcv_msg+0x609/0x860 net/core/rtnetlink.c:4104
>  netlink_rcv_skb+0x2ab/0x390 net/netlink/af_netlink.c:2298
>  rtnetlink_rcv+0x2a/0x40 net/core/rtnetlink.c:4110
>  netlink_unicast_kernel net/netlink/af_netlink.c:1231 [inline]
>  netlink_unicast+0x525/0x730 net/netlink/af_netlink.c:1257
>  netlink_sendmsg+0xab3/0xe70 net/netlink/af_netlink.c:1803
>  sock_sendmsg_nosec net/socket.c:633 [inline]
>  sock_sendmsg+0xca/0x110 net/socket.c:643
>  sock_write_iter+0x326/0x600 net/socket.c:846
>  call_write_iter include/linux/fs.h:1733 [inline]
>  do_iter_readv_writev fs/read_write.c:696 [inline]
>  __do_readv_writev+0xbbc/0x10a0 fs/read_write.c:862
>  do_readv_writev+0x13f/0x200 fs/read_write.c:894
>  vfs_writev+0x87/0xc0 fs/read_write.c:921
>  do_writev+0x110/0x2c0 fs/read_write.c:954
>  SYSC_writev fs/read_write.c:1027 [inline]
>  SyS_writev+0x27/0x30 fs/read_write.c:1024
>  entry_SYSCALL_64_fastpath+0x1f/0xc2
> RIP: 0033:0x4458d9
> RSP: 002b:00007f1639006b58 EFLAGS: 00000292 ORIG_RAX: 0000000000000014
> RAX: ffffffffffffffda RBX: 0000000000000019 RCX: 00000000004458d9
> RDX: 0000000000000001 RSI: 00000000207cd000 RDI: 0000000000000019
> RBP: 00000000006e30c0 R08: 0000000000000000 R09: 0000000000000000
> R10: 0000000000000000 R11: 0000000000000292 R12: 0000000000708000
> R13: 0000000000000010 R14: 0000000000000003 R15: 0000000000000000



I've commented that warning just to see I can obtain more information.
Then I also got this:

------------[ cut here ]------------
WARNING: CPU: 2 PID: 3990 at net/ipv6/ip6_fib.c:991
fib6_add+0x2e12/0x3290 net/ipv6/ip6_fib.c:991 net/ipv6/ip6_fib.c:991
Kernel panic - not syncing: panic_on_warn set ...

CPU: 2 PID: 3990 Comm: kworker/2:4 Not tainted 4.11.0-rc1+ #311
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Workqueue: ipv6_addrconf addrconf_dad_work
Call Trace:
 __dump_stack lib/dump_stack.c:16 [inline]
 __dump_stack lib/dump_stack.c:16 [inline] lib/dump_stack.c:52
 dump_stack+0x2fb/0x3fd lib/dump_stack.c:52 lib/dump_stack.c:52
 panic+0x20f/0x426 kernel/panic.c:180 kernel/panic.c:180
 __warn+0x1c4/0x1e0 kernel/panic.c:541 kernel/panic.c:541
 warn_slowpath_null+0x2c/0x40 kernel/panic.c:584 kernel/panic.c:584
 fib6_add+0x2e12/0x3290 net/ipv6/ip6_fib.c:991 net/ipv6/ip6_fib.c:991
 __ip6_ins_rt+0x60/0x80 net/ipv6/route.c:948 net/ipv6/route.c:948
 ip6_ins_rt+0x19b/0x220 net/ipv6/route.c:959 net/ipv6/route.c:959
 __ipv6_ifa_notify+0x62e/0x7a0 net/ipv6/addrconf.c:5485 net/ipv6/addrconf.c:5485
 ipv6_ifa_notify+0xdf/0x1d0 net/ipv6/addrconf.c:5518 net/ipv6/addrconf.c:5518
 addrconf_dad_completed+0xe6/0x950 net/ipv6/addrconf.c:3983
net/ipv6/addrconf.c:3983
 addrconf_dad_begin net/ipv6/addrconf.c:3797 [inline]
 addrconf_dad_begin net/ipv6/addrconf.c:3797 [inline] net/ipv6/addrconf.c:3897
 addrconf_dad_work+0x32a/0xea0 net/ipv6/addrconf.c:3897 net/ipv6/addrconf.c:3897
 process_one_work+0xc06/0x1c40 kernel/workqueue.c:2096 kernel/workqueue.c:2096
 worker_thread+0x223/0x19f0 kernel/workqueue.c:2230 kernel/workqueue.c:2230
 kthread+0x334/0x400 kernel/kthread.c:229 kernel/kthread.c:229
 ret_from_fork+0x31/0x40 arch/x86/entry/entry_64.S:430
arch/x86/entry/entry_64.S:430



And this without any preceding warnings:

==================================================================
BUG: KASAN: slab-out-of-bounds in fib6_age+0x3fd/0x480
net/ipv6/ip6_fib.c:1787 at addr ffff88004d4fbe54
Read of size 4 by task swapper/2/0
CPU: 2 PID: 0 Comm: swapper/2 Not tainted 4.11.0-rc1+ #311
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
 <IRQ>
 __dump_stack lib/dump_stack.c:16 [inline]
 dump_stack+0x2fb/0x3fd lib/dump_stack.c:52
 kasan_object_err+0x1c/0x90 mm/kasan/report.c:166
 print_address_description mm/kasan/report.c:208 [inline]
 kasan_report_error mm/kasan/report.c:292 [inline]
 kasan_report.part.2+0x1b0/0x460 mm/kasan/report.c:314
 kasan_report mm/kasan/report.c:334 [inline]
 __asan_report_load4_noabort+0x29/0x30 mm/kasan/report.c:334
 fib6_age+0x3fd/0x480 net/ipv6/ip6_fib.c:1787
 fib6_clean_node+0x356/0x550 net/ipv6/ip6_fib.c:1665
 fib6_walk_continue+0x4b3/0x620 net/ipv6/ip6_fib.c:1594
 fib6_walk+0x91/0xf0 net/ipv6/ip6_fib.c:1639
 fib6_clean_tree+0x266/0x3a0 net/ipv6/ip6_fib.c:1711
 __fib6_clean_all+0x1e1/0x360 net/ipv6/ip6_fib.c:1727
 fib6_clean_all net/ipv6/ip6_fib.c:1738 [inline]
 fib6_run_gc+0x185/0x3d0 net/ipv6/ip6_fib.c:1835
 fib6_gc_timer_cb+0x1c/0x20 net/ipv6/ip6_fib.c:1850
 call_timer_fn+0x241/0x820 kernel/time/timer.c:1268
 expire_timers kernel/time/timer.c:1307 [inline]
 __run_timers+0x960/0xcf0 kernel/time/timer.c:1601
 run_timer_softirq+0x21/0x80 kernel/time/timer.c:1614
 __do_softirq+0x31f/0xbe7 kernel/softirq.c:284
 invoke_softirq kernel/softirq.c:364 [inline]
 irq_exit+0x1cc/0x200 kernel/softirq.c:405
 exiting_irq arch/x86/include/asm/apic.h:657 [inline]
 smp_apic_timer_interrupt+0x76/0xa0 arch/x86/kernel/apic/apic.c:962
 apic_timer_interrupt+0x93/0xa0 arch/x86/entry/entry_64.S:487
RIP: 0010:native_safe_halt+0x6/0x10 arch/x86/include/asm/irqflags.h:53
RSP: 0018:ffff880089437c10 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff10
RAX: dffffc0000000000 RBX: 1ffff10011286f85 RCX: 0000000000000000
RDX: 1ffffffff0a18ebc RSI: 0000000000000001 RDI: ffffffff850c75e0
RBP: ffff880089437c10 R08: ffffed00113835c2 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 1ffff10011286fa9
R13: ffff880089437cc8 R14: ffffffff856973f8 R15: ffff880089437e68
 </IRQ>
 arch_safe_halt arch/x86/include/asm/paravirt.h:98 [inline]
 default_idle+0xbf/0x440 arch/x86/kernel/process.c:275
 arch_cpu_idle+0xa/0x10 arch/x86/kernel/process.c:266
 default_idle_call+0x36/0x90 kernel/sched/idle.c:97
 cpuidle_idle_call kernel/sched/idle.c:155 [inline]
 do_idle+0x373/0x520 kernel/sched/idle.c:244
 cpu_startup_entry+0x18/0x20 kernel/sched/idle.c:346
 start_secondary+0x36c/0x460 arch/x86/kernel/smpboot.c:275
 start_cpu+0x14/0x14 arch/x86/kernel/head_64.S:306
Object at ffff88004d4fbd40, in cache ip_dst_cache size: 216
Allocated:
PID = 8122
 save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:59
 save_stack+0x43/0xd0 mm/kasan/kasan.c:513
 set_track mm/kasan/kasan.c:525 [inline]
 kasan_kmalloc+0xaa/0xd0 mm/kasan/kasan.c:616
 kasan_slab_alloc+0x12/0x20 mm/kasan/kasan.c:555
 kmem_cache_alloc+0x102/0x6e0 mm/slab.c:3572
 dst_alloc+0x11b/0x1a0 net/core/dst.c:209
 rt_dst_alloc+0xf0/0x580 net/ipv4/route.c:1482
 __mkroute_output net/ipv4/route.c:2165 [inline]
 __ip_route_output_key_hash+0xce3/0x2ca0 net/ipv4/route.c:2375
 __ip_route_output_key include/net/route.h:122 [inline]
 ip_route_output_flow+0x29/0xa0 net/ipv4/route.c:2461
 ip_route_output_key include/net/route.h:132 [inline]
 sctp_v4_get_dst+0x5d2/0x1570 net/sctp/protocol.c:458
 sctp_transport_route+0xa8/0x420 net/sctp/transport.c:292
 sctp_assoc_add_peer+0x5a5/0x1470 net/sctp/associola.c:653
 sctp_sendmsg+0x180d/0x3980 net/sctp/socket.c:1871
 inet_sendmsg+0x164/0x5b0 net/ipv4/af_inet.c:761
 sock_sendmsg_nosec net/socket.c:633 [inline]
 sock_sendmsg+0xca/0x110 net/socket.c:643
 SYSC_sendto+0x660/0x810 net/socket.c:1685
 SyS_sendto+0x40/0x50 net/socket.c:1653
 entry_SYSCALL_64_fastpath+0x1f/0xc2
Freed:
PID = 2038
 save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:59
 save_stack+0x43/0xd0 mm/kasan/kasan.c:513
 set_track mm/kasan/kasan.c:525 [inline]
 kasan_slab_free+0x6f/0xb0 mm/kasan/kasan.c:589
 __cache_free mm/slab.c:3514 [inline]
 kmem_cache_free+0x71/0x240 mm/slab.c:3774
 dst_destroy+0x211/0x340 net/core/dst.c:272
 dst_free include/net/dst.h:429 [inline]
 dst_rcu_free+0x152/0x190 include/net/dst.h:439
 __rcu_reclaim kernel/rcu/rcu.h:118 [inline]
 rcu_do_batch.isra.66+0xa31/0xe50 kernel/rcu/tree.c:2880
 invoke_rcu_callbacks kernel/rcu/tree.c:3143 [inline]
 __rcu_process_callbacks kernel/rcu/tree.c:3110 [inline]
 rcu_process_callbacks+0x45b/0xc50 kernel/rcu/tree.c:3127
 __do_softirq+0x31f/0xbe7 kernel/softirq.c:284
Disposed:
PID = 26270
 save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:59
 save_stack+0x43/0xd0 mm/kasan/kasan.c:513
 set_track mm/kasan/kasan.c:525 [inline]
 kasan_set_rcu_track+0xcf/0xf0 mm/kasan/kasan.c:694
 __call_rcu.constprop.77+0x1d6/0x15a0 kernel/rcu/tree.c:3230
 call_rcu_sched+0x12/0x20 kernel/rcu/tree.c:3291
 rt_free net/ipv4/route.c:592 [inline]
 rt_cache_route+0xf5/0x130 net/ipv4/route.c:1365
 rt_set_nexthop.constprop.57+0x408/0xfa0 net/ipv4/route.c:1453
 __mkroute_output net/ipv4/route.c:2195 [inline]
 __ip_route_output_key_hash+0xe50/0x2ca0 net/ipv4/route.c:2375
 __ip_route_output_key include/net/route.h:122 [inline]
 ip_route_output_flow+0x29/0xa0 net/ipv4/route.c:2461
 ip_route_output_key include/net/route.h:132 [inline]
 sctp_v4_get_dst+0x5d2/0x1570 net/sctp/protocol.c:458
 sctp_transport_route+0xa8/0x420 net/sctp/transport.c:292
 sctp_assoc_add_peer+0x5a5/0x1470 net/sctp/associola.c:653
 sctp_process_param net/sctp/sm_make_chunk.c:2548 [inline]
 sctp_process_init+0xf71/0x2320 net/sctp/sm_make_chunk.c:2354
 sctp_sf_do_unexpected_init.isra.28+0x7b8/0x1470 net/sctp/sm_statefuns.c:1510
 sctp_sf_do_5_2_1_siminit+0x35/0x40 net/sctp/sm_statefuns.c:1199
 sctp_do_sm+0x1e5/0x6a30 net/sctp/sm_sideeffect.c:1144
 sctp_assoc_bh_rcv+0x285/0x4b0 net/sctp/associola.c:1063
 sctp_inq_push+0x22b/0x2e0 net/sctp/inqueue.c:95
 sctp_backlog_rcv+0x177/0xb40 net/sctp/input.c:350
 sk_backlog_rcv include/net/sock.h:896 [inline]
 __release_sock+0x126/0x3a0 net/core/sock.c:2058
 release_sock+0xa5/0x2b0 net/core/sock.c:2545
 sctp_sendmsg+0x2b05/0x3980 net/sctp/socket.c:2011
 inet_sendmsg+0x164/0x5b0 net/ipv4/af_inet.c:761
 sock_sendmsg_nosec net/socket.c:633 [inline]
 sock_sendmsg+0xca/0x110 net/socket.c:643
 SYSC_sendto+0x660/0x810 net/socket.c:1685
 SyS_sendto+0x40/0x50 net/socket.c:1653
 entry_SYSCALL_64_fastpath+0x1f/0xc2
Memory state around the buggy address:
 ffff88004d4fbd00: fc fc fc fc fc fc fc fc 00 00 00 00 00 00 00 00
 ffff88004d4fbd80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>ffff88004d4fbe00: 00 00 00 fc fc fc fc fc fc fc fc fc fc fc fc fc
                                                 ^
 ffff88004d4fbe80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 ffff88004d4fbf00: 00 00 00 00 00 00 00 00 00 00 00 fc fc fc fc fc
==================================================================

[toc] | [prev] | [next] | [standalone]

#1594493

On Tue, Mar 7, 2017 at 7:03 PM, David Ahern <dsa@cumulusnetworks.com> wrote:
> On 3/7/17 2:21 AM, Dmitry Vyukov wrote:
>> I've commented that warning just to see I can obtain more information.
>> Then I also got this:
>>
>> ------------[ cut here ]------------
>> WARNING: CPU: 2 PID: 3990 at net/ipv6/ip6_fib.c:991
>> fib6_add+0x2e12/0x3290 net/ipv6/ip6_fib.c:991 net/ipv6/ip6_fib.c:991
>> Kernel panic - not syncing: panic_on_warn set ...
>
> again panic_on_warn is triggering ...
>
>>
>> CPU: 2 PID: 3990 Comm: kworker/2:4 Not tainted 4.11.0-rc1+ #311
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>> Workqueue: ipv6_addrconf addrconf_dad_work
>> Call Trace:
>>  __dump_stack lib/dump_stack.c:16 [inline]
>>  __dump_stack lib/dump_stack.c:16 [inline] lib/dump_stack.c:52
>>  dump_stack+0x2fb/0x3fd lib/dump_stack.c:52 lib/dump_stack.c:52
>>  panic+0x20f/0x426 kernel/panic.c:180 kernel/panic.c:180
>>  __warn+0x1c4/0x1e0 kernel/panic.c:541 kernel/panic.c:541
>>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:584 kernel/panic.c:584
>>  fib6_add+0x2e12/0x3290 net/ipv6/ip6_fib.c:991 net/ipv6/ip6_fib.c:991
>
> on this warning:
>
> /* dst.next really should not be set at this point */
> if (rt->dst.next && rt->dst.next->ops->family != AF_INET6) {
>         pr_warn("fib6_add: adding rt with bad next -- family %d dst
> flags %x\n",
>                 rt->dst.next->ops->family, rt->dst.next->flags);
>
>         WARN_ON(1);
> }
>
> You should have seen the pr_warn in the log preceding the WARN_ON dump.


Right. They all have the same "IPv6: fib6_add: adding rt with bad next
-- family 2 dst flags 6"

[  171.222795] IPv6: fib6_add: adding rt with bad next -- family 2 dst flags 6
[  171.223809] ------------[ cut here ]------------
[  171.224407] WARNING: CPU: 3 PID: 27 at net/ipv6/ip6_fib.c:991
fib6_add+0x2e12/0x3290
[  171.225327] Kernel panic - not syncing: panic_on_warn set ...
[  171.225327]
[  171.226066] CPU: 3 PID: 27 Comm: kworker/3:0 Not tainted 4.11.0-rc1+ #311
[  171.226304] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS Bochs 01/01/2011
[  171.226304] Workqueue: ipv6_addrconf addrconf_dad_work
[  171.226304] Call Trace:
[  171.226304]  dump_stack+0x2fb/0x3fd
[  171.226304]  ? arch_local_irq_restore+0x53/0x53
[  171.226304]  ? vprintk_emit+0x566/0x770
[  171.226304]  ? console_unlock+0xf50/0xf50
[  171.226304]  ? vprintk_emit+0x566/0x770
[  171.226304]  ? console_unlock+0xf50/0xf50
[  171.226304]  ? vprintk_emit+0x566/0x770
[  171.226304]  ? console_unlock+0xf50/0xf50
[  171.226304]  ? check_noncircular+0x20/0x20
[  171.226304]  ? trace_hardirqs_on+0xd/0x10
[  171.226304]  ? perf_trace_lock_acquire+0x141/0xa00
[  171.226304]  ? trace_hardirqs_off+0xd/0x10
[  171.226304]  ? quarantine_put+0xea/0x190
[  171.226304]  ? check_noncircular+0x20/0x20
[  171.236060]  ? vprintk_default+0x28/0x30
[  171.236662]  ? vprintk_func+0x47/0x90
[  171.236662]  ? printk+0xc8/0xf9
[  171.236662]  ? load_image_and_restore+0x134/0x134
[  171.236662]  ? pointer+0xac0/0xac0
[  171.236662]  panic+0x20f/0x426
[  171.236662]  ? copy_mm+0x1219/0x1219
[  171.236662]  ? vprintk_func+0x47/0x90
[  171.236662]  ? printk+0xc8/0xf9
[  171.236662]  ? fib6_add+0x2e12/0x3290
[  171.236662]  __warn+0x1c4/0x1e0
[  171.236662]  warn_slowpath_null+0x2c/0x40
[  171.236662]  fib6_add+0x2e12/0x3290
[  171.236662]  ? kasan_check_write+0x14/0x20
[  171.236662]  ? netlink_broadcast_filtered+0x734/0x1380
[  171.236662]  ? fib6_force_start_gc+0xf0/0xf0
[  171.236662]  ? netlink_has_listeners+0x450/0x450
[  171.236662]  ? memcpy+0x45/0x50
[  171.236662]  ? __nla_put+0x37/0x40
[  171.236662]  ? nla_put+0xf9/0x130
[  171.236662]  ? skb_put+0x149/0x1c0
[  171.236662]  ? kasan_check_write+0x14/0x20
[  171.236662]  ? do_raw_write_lock+0xbd/0x1e0
[  171.236662]  __ip6_ins_rt+0x60/0x80
[  171.236662]  ip6_ins_rt+0x19b/0x220
[  171.236662]  ? ip6_route_info_create+0x2380/0x2380
[  171.236662]  ? nlmsg_notify+0xaf/0x160
[  171.236662]  ? rtnl_notify+0xbb/0xe0
[  171.236662]  __ipv6_ifa_notify+0x62e/0x7a0
[  171.251057]  ipv6_ifa_notify+0xdf/0x1d0
[  171.251057]  ? __ipv6_ifa_notify+0x7a0/0x7a0
[  171.251057]  addrconf_dad_completed+0xe6/0x950
[  171.251057]  ? addrconf_verify_work+0x20/0x20
[  171.251057]  ? kasan_check_write+0x14/0x20
[  171.251057]  addrconf_dad_work+0x32a/0xea0
[  171.251057]  ? addrconf_ifdown+0x1ad0/0x1ad0
[  171.251057]  ? rcu_pm_notify+0xc0/0xc0
[  171.251057]  ? wq_update_unbound_numa+0x8d0/0x8d0
[  171.251057]  ? kasan_check_write+0x14/0x20
[  171.251057]  process_one_work+0xc06/0x1c40
[  171.251057]  ? process_one_work+0xb3d/0x1c40
[  171.251057]  ? pwq_dec_nr_in_flight+0x470/0x470
[  171.251057]  ? preempt_notifier_register+0x1f0/0x1f0
[  171.259856]  ? __schedule+0x893/0x22d0
[  171.259856]  ? kasan_check_write+0x14/0x20
[  171.259856]  ? worker_thread+0x47d/0x19f0
[  171.259856]  ? lock_set_class+0xc00/0xc00
[  171.259856]  ? worker_thread+0x467/0x19f0
[  171.259856]  ? lock_acquire+0x630/0x630
[  171.259856]  ? _raw_spin_unlock_irq+0x27/0x70
[  171.259856]  ? check_noncircular+0x20/0x20
[  171.259856]  ? mark_held_locks+0x100/0x100
[  171.259856]  ? trace_hardirqs_on_thunk+0x1a/0x1c
[  171.259856]  ? __schedule+0x22d0/0x22d0
[  171.259856]  ? do_raw_spin_trylock+0x1a0/0x1a0
[  171.259856]  ? do_raw_spin_lock+0xbd/0x1f0
[  171.259856]  worker_thread+0x223/0x19f0
[  171.259856]  ? process_one_work+0x1c40/0x1c40
[  171.259856]  ? lock_repin_lock+0x4a0/0x4a0
[  171.259856]  ? unwind_dump.isra.5.part.6+0x320/0x320
[  171.259856]  ? kasan_check_write+0x14/0x20
[  171.259856]  ? finish_task_switch+0x1ea/0x740
[  171.259856]  ? finish_task_switch+0x196/0x740
[  171.259856]  ? preempt_notifier_register+0x1f0/0x1f0
[  171.259856]  ? __schedule+0x893/0x22d0
[  171.259856]  ? lockdep_count_backward_deps+0x480/0x480
[  171.259856]  ? ret_from_fork+0x31/0x40
[  171.259856]  ? do_raw_spin_lock+0xbd/0x1f0
[  171.259856]  ? complete+0xbf/0x190
[  171.259856]  ? register_lock_class+0x1c30/0x1c30
[  171.276560]  ? __wake_up_common+0xb4/0x150
[  171.276560]  ? rcu_pm_notify+0xc0/0xc0
[  171.276560]  ? __schedule+0x22d0/0x22d0
[  171.276560]  ? __init_waitqueue_head+0x8a/0x120
[  171.276560]  ? __wake_up_bit+0x290/0x290
[  171.279715]  ? preempt_notifier_register+0x1f0/0x1f0
[  171.279715]  ? __kthread_parkme+0x173/0x240
[  171.279715]  kthread+0x334/0x400
[  171.279715]  ? process_one_work+0x1c40/0x1c40
[  171.279715]  ? kthread_create_on_node+0x110/0x110
[  171.279715]  ret_from_fork+0x31/0x40
[  171.279715] Dumping ftrace buffer:
[  171.279715]    (ftrace buffer empty)
[  171.279715] Kernel Offset: disabled
[  171.279715] Rebooting in 86400 seconds..




>>  __ip6_ins_rt+0x60/0x80 net/ipv6/route.c:948 net/ipv6/route.c:948
>>  ip6_ins_rt+0x19b/0x220 net/ipv6/route.c:959 net/ipv6/route.c:959
>>  __ipv6_ifa_notify+0x62e/0x7a0 net/ipv6/addrconf.c:5485 net/ipv6/addrconf.c:5485
>>  ipv6_ifa_notify+0xdf/0x1d0 net/ipv6/addrconf.c:5518 net/ipv6/addrconf.c:5518
>>  addrconf_dad_completed+0xe6/0x950 net/ipv6/addrconf.c:3983
>> net/ipv6/addrconf.c:3983
>>  addrconf_dad_begin net/ipv6/addrconf.c:3797 [inline]
>>  addrconf_dad_begin net/ipv6/addrconf.c:3797 [inline] net/ipv6/addrconf.c:3897
>>  addrconf_dad_work+0x32a/0xea0 net/ipv6/addrconf.c:3897 net/ipv6/addrconf.c:3897
>>  process_one_work+0xc06/0x1c40 kernel/workqueue.c:2096 kernel/workqueue.c:2096
>>  worker_thread+0x223/0x19f0 kernel/workqueue.c:2230 kernel/workqueue.c:2230
>>  kthread+0x334/0x400 kernel/kthread.c:229 kernel/kthread.c:229
>>  ret_from_fork+0x31/0x40 arch/x86/entry/entry_64.S:430
>> arch/x86/entry/entry_64.S:430
>>
>>
>>
>> And this without any preceding warnings:
>>
>> ==================================================================
>> BUG: KASAN: slab-out-of-bounds in fib6_age+0x3fd/0x480
>> net/ipv6/ip6_fib.c:1787 at addr ffff88004d4fbe54
>
> another ipv4 route in ipv6 fib walk

[toc] | [prev] | [next] | [standalone]

#1594517 — Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone

[Multipart message — attachments visible in raw view] — view raw

On 3/7/17 11:13 AM, Dmitry Vyukov wrote:
>> on this warning:
>>
>> /* dst.next really should not be set at this point */
>> if (rt->dst.next && rt->dst.next->ops->family != AF_INET6) {
>>         pr_warn("fib6_add: adding rt with bad next -- family %d dst
>> flags %x\n",
>>                 rt->dst.next->ops->family, rt->dst.next->flags);
>>
>>         WARN_ON(1);
>> }
>>
>> You should have seen the pr_warn in the log preceding the WARN_ON dump.
> 
> Right. They all have the same "IPv6: fib6_add: adding rt with bad next
> -- family 2 dst flags 6"

remove the previous changes and try the attached.

[toc] | [prev] | [next] | [standalone]

#1594552

On Tue, Mar 7, 2017 at 7:43 PM, David Ahern <dsa@cumulusnetworks.com> wrote:
> On 3/7/17 11:13 AM, Dmitry Vyukov wrote:
>>> on this warning:
>>>
>>> /* dst.next really should not be set at this point */
>>> if (rt->dst.next && rt->dst.next->ops->family != AF_INET6) {
>>>         pr_warn("fib6_add: adding rt with bad next -- family %d dst
>>> flags %x\n",
>>>                 rt->dst.next->ops->family, rt->dst.next->flags);
>>>
>>>         WARN_ON(1);
>>> }
>>>
>>> You should have seen the pr_warn in the log preceding the WARN_ON dump.
>>
>> Right. They all have the same "IPv6: fib6_add: adding rt with bad next
>> -- family 2 dst flags 6"
>
> remove the previous changes and try the attached.


Doing this now.
FWIW I've also applied your last patch with missing "iter->dst.flags
&= ~DST_IN_FIB;" and restored the warning in rt6_rcu_free and it did
not fire (in a limited run). I only saw the "WARNING in fib6_add" that
I already reported.

[toc] | [prev] | [next] | [standalone]

#1594589

On Tue, Mar 7, 2017 at 8:02 PM, Dmitry Vyukov <dvyukov@google.com> wrote:
> On Tue, Mar 7, 2017 at 7:43 PM, David Ahern <dsa@cumulusnetworks.com> wrote:
>> On 3/7/17 11:13 AM, Dmitry Vyukov wrote:
>>>> on this warning:
>>>>
>>>> /* dst.next really should not be set at this point */
>>>> if (rt->dst.next && rt->dst.next->ops->family != AF_INET6) {
>>>>         pr_warn("fib6_add: adding rt with bad next -- family %d dst
>>>> flags %x\n",
>>>>                 rt->dst.next->ops->family, rt->dst.next->flags);
>>>>
>>>>         WARN_ON(1);
>>>> }
>>>>
>>>> You should have seen the pr_warn in the log preceding the WARN_ON dump.
>>>
>>> Right. They all have the same "IPv6: fib6_add: adding rt with bad next
>>> -- family 2 dst flags 6"
>>
>> remove the previous changes and try the attached.
>
>
> Doing this now.
> FWIW I've also applied your last patch with missing "iter->dst.flags
> &= ~DST_IN_FIB;" and restored the warning in rt6_rcu_free and it did
> not fire (in a limited run). I only saw the "WARNING in fib6_add" that
> I already reported.


So far I've hit only:
[ 1103.840031] BUG: KASAN: slab-out-of-bounds in fib6_age+0x3fd/0x480
at addr ffff8800799d2254
without any preceeding warnings.
But note that since the kernel is heavily stressed I can reliably get
any pr_err output if it happens right before BUG/WARNING. Anything
that happens minutes before will be lots because there are tons of
output.

[toc] | [prev] | [next] | [standalone]

#1594601

On Tue, Mar 7, 2017 at 8:30 PM, Dmitry Vyukov <dvyukov@google.com> wrote:
>>> On 3/7/17 11:13 AM, Dmitry Vyukov wrote:
>>>>> on this warning:
>>>>>
>>>>> /* dst.next really should not be set at this point */
>>>>> if (rt->dst.next && rt->dst.next->ops->family != AF_INET6) {
>>>>>         pr_warn("fib6_add: adding rt with bad next -- family %d dst
>>>>> flags %x\n",
>>>>>                 rt->dst.next->ops->family, rt->dst.next->flags);
>>>>>
>>>>>         WARN_ON(1);
>>>>> }
>>>>>
>>>>> You should have seen the pr_warn in the log preceding the WARN_ON dump.
>>>>
>>>> Right. They all have the same "IPv6: fib6_add: adding rt with bad next
>>>> -- family 2 dst flags 6"
>>>
>>> remove the previous changes and try the attached.
>>
>>
>> Doing this now.
>> FWIW I've also applied your last patch with missing "iter->dst.flags
>> &= ~DST_IN_FIB;" and restored the warning in rt6_rcu_free and it did
>> not fire (in a limited run). I only saw the "WARNING in fib6_add" that
>> I already reported.
>
>
> So far I've hit only:
> [ 1103.840031] BUG: KASAN: slab-out-of-bounds in fib6_age+0x3fd/0x480
> at addr ffff8800799d2254
> without any preceeding warnings.
> But note that since the kernel is heavily stressed I can reliably get
> any pr_err output if it happens right before BUG/WARNING. Anything
> that happens minutes before will be lots because there are tons of
> output.



So far 6 "KASAN: slab-out-of-bounds Read in fib6_age" but no other warnings.

[toc] | [prev] | [next] | [standalone]

#1595114

On Tue, Mar 7, 2017 at 9:00 PM, Dmitry Vyukov <dvyukov@google.com> wrote:
> On Tue, Mar 7, 2017 at 8:30 PM, Dmitry Vyukov <dvyukov@google.com> wrote:
>>>> On 3/7/17 11:13 AM, Dmitry Vyukov wrote:
>>>>>> on this warning:
>>>>>>
>>>>>> /* dst.next really should not be set at this point */
>>>>>> if (rt->dst.next && rt->dst.next->ops->family != AF_INET6) {
>>>>>>         pr_warn("fib6_add: adding rt with bad next -- family %d dst
>>>>>> flags %x\n",
>>>>>>                 rt->dst.next->ops->family, rt->dst.next->flags);
>>>>>>
>>>>>>         WARN_ON(1);
>>>>>> }
>>>>>>
>>>>>> You should have seen the pr_warn in the log preceding the WARN_ON dump.
>>>>>
>>>>> Right. They all have the same "IPv6: fib6_add: adding rt with bad next
>>>>> -- family 2 dst flags 6"
>>>>
>>>> remove the previous changes and try the attached.
>>>
>>>
>>> Doing this now.
>>> FWIW I've also applied your last patch with missing "iter->dst.flags
>>> &= ~DST_IN_FIB;" and restored the warning in rt6_rcu_free and it did
>>> not fire (in a limited run). I only saw the "WARNING in fib6_add" that
>>> I already reported.
>>
>>
>> So far I've hit only:
>> [ 1103.840031] BUG: KASAN: slab-out-of-bounds in fib6_age+0x3fd/0x480
>> at addr ffff8800799d2254
>> without any preceeding warnings.
>> But note that since the kernel is heavily stressed I can reliably get
>> any pr_err output if it happens right before BUG/WARNING. Anything
>> that happens minutes before will be lots because there are tons of
>> output.
>
>
>
> So far 6 "KASAN: slab-out-of-bounds Read in fib6_age" but no other warnings.


I've got a bunch of the crashes that I was getting previously, but no
new warnings.

[toc] | [prev] | [next] | [standalone]

#1594849 — Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone

On 3/7/17 2:21 AM, Dmitry Vyukov wrote:
> I've commented that warning just to see I can obtain more information.
> Then I also got this:
> 
> ------------[ cut here ]------------
> WARNING: CPU: 2 PID: 3990 at net/ipv6/ip6_fib.c:991
> fib6_add+0x2e12/0x3290 net/ipv6/ip6_fib.c:991 net/ipv6/ip6_fib.c:991
> Kernel panic - not syncing: panic_on_warn set ...

again panic_on_warn is triggering ...

> 
> CPU: 2 PID: 3990 Comm: kworker/2:4 Not tainted 4.11.0-rc1+ #311
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Workqueue: ipv6_addrconf addrconf_dad_work
> Call Trace:
>  __dump_stack lib/dump_stack.c:16 [inline]
>  __dump_stack lib/dump_stack.c:16 [inline] lib/dump_stack.c:52
>  dump_stack+0x2fb/0x3fd lib/dump_stack.c:52 lib/dump_stack.c:52
>  panic+0x20f/0x426 kernel/panic.c:180 kernel/panic.c:180
>  __warn+0x1c4/0x1e0 kernel/panic.c:541 kernel/panic.c:541
>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:584 kernel/panic.c:584
>  fib6_add+0x2e12/0x3290 net/ipv6/ip6_fib.c:991 net/ipv6/ip6_fib.c:991

on this warning:

/* dst.next really should not be set at this point */
if (rt->dst.next && rt->dst.next->ops->family != AF_INET6) {
        pr_warn("fib6_add: adding rt with bad next -- family %d dst
flags %x\n",
                rt->dst.next->ops->family, rt->dst.next->flags);

        WARN_ON(1);
}

You should have seen the pr_warn in the log preceding the WARN_ON dump.


>  __ip6_ins_rt+0x60/0x80 net/ipv6/route.c:948 net/ipv6/route.c:948
>  ip6_ins_rt+0x19b/0x220 net/ipv6/route.c:959 net/ipv6/route.c:959
>  __ipv6_ifa_notify+0x62e/0x7a0 net/ipv6/addrconf.c:5485 net/ipv6/addrconf.c:5485
>  ipv6_ifa_notify+0xdf/0x1d0 net/ipv6/addrconf.c:5518 net/ipv6/addrconf.c:5518
>  addrconf_dad_completed+0xe6/0x950 net/ipv6/addrconf.c:3983
> net/ipv6/addrconf.c:3983
>  addrconf_dad_begin net/ipv6/addrconf.c:3797 [inline]
>  addrconf_dad_begin net/ipv6/addrconf.c:3797 [inline] net/ipv6/addrconf.c:3897
>  addrconf_dad_work+0x32a/0xea0 net/ipv6/addrconf.c:3897 net/ipv6/addrconf.c:3897
>  process_one_work+0xc06/0x1c40 kernel/workqueue.c:2096 kernel/workqueue.c:2096
>  worker_thread+0x223/0x19f0 kernel/workqueue.c:2230 kernel/workqueue.c:2230
>  kthread+0x334/0x400 kernel/kthread.c:229 kernel/kthread.c:229
>  ret_from_fork+0x31/0x40 arch/x86/entry/entry_64.S:430
> arch/x86/entry/entry_64.S:430
> 
> 
> 
> And this without any preceding warnings:
> 
> ==================================================================
> BUG: KASAN: slab-out-of-bounds in fib6_age+0x3fd/0x480
> net/ipv6/ip6_fib.c:1787 at addr ffff88004d4fbe54

another ipv4 route in ipv6 fib walk

[toc] | [prev] | [next] | [standalone]

#1594459

On Tue, Mar 7, 2017 at 6:17 PM, 'David Ahern' via syzkaller
<syzkaller@googlegroups.com> wrote:
> On 3/7/17 1:43 AM, Dmitry Vyukov wrote:
>> This is on c1ae3cfa0e89fa1a7ecc4c99031f5e9ae99d9201. No other kernel
>> output from your patch (pr_err).
>
> Is the below supposed to be from the same qemu instance at the time of
> the crash? cpu1 and cpu2 are both supposedly doing a route insert?


No, it's all from different instances.

>> ------------[ cut here ]------------
>> WARNING: CPU: 1 PID: 30179 at net/ipv6/ip6_fib.c:158
>> rt6_rcu_free+0x61/0x70 net/ipv6/ip6_fib.c:158
>> Kernel panic - not syncing: panic_on_warn set ...
>>
>> CPU: 1 PID: 30179 Comm: syz-executor3 Not tainted 4.11.0-rc1+ #310
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>> Call Trace:
>>  __dump_stack lib/dump_stack.c:16 [inline]
>>  dump_stack+0x2fb/0x3fd lib/dump_stack.c:52
>>  panic+0x20f/0x426 kernel/panic.c:180
>>  __warn+0x1c4/0x1e0 kernel/panic.c:541
>>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:584
>>  rt6_rcu_free+0x61/0x70 net/ipv6/ip6_fib.c:158
>>  rt6_release+0x1ee/0x290 net/ipv6/ip6_fib.c:189
>>  fib6_add_rt2node net/ipv6/ip6_fib.c:922 [inline]
>>  fib6_add+0x1d51/0x3290 net/ipv6/ip6_fib.c:1081
>>  __ip6_ins_rt+0x60/0x80 net/ipv6/route.c:948
>>  ip6_route_add+0x1a7/0x310 net/ipv6/route.c:2130
>>  inet6_rtm_newroute+0x191/0x1b0 net/ipv6/route.c:3294
>>  rtnetlink_rcv_msg+0x609/0x860 net/core/rtnetlink.c:4104
>>  netlink_rcv_skb+0x2ab/0x390 net/netlink/af_netlink.c:2298
>>  rtnetlink_rcv+0x2a/0x40 net/core/rtnetlink.c:4110
>>  netlink_unicast_kernel net/netlink/af_netlink.c:1231 [inline]
>>  netlink_unicast+0x525/0x730 net/netlink/af_netlink.c:1257
>>  netlink_sendmsg+0xab3/0xe70 net/netlink/af_netlink.c:1803
>>  sock_sendmsg_nosec net/socket.c:633 [inline]
>>  sock_sendmsg+0xca/0x110 net/socket.c:643
>>  sock_write_iter+0x326/0x600 net/socket.c:846
>>  call_write_iter include/linux/fs.h:1733 [inline]
>>  do_iter_readv_writev fs/read_write.c:696 [inline]
>>  __do_readv_writev+0xbbc/0x10a0 fs/read_write.c:862
>>  do_readv_writev+0x13f/0x200 fs/read_write.c:894
>>  vfs_writev+0x87/0xc0 fs/read_write.c:921
>>  do_writev+0x110/0x2c0 fs/read_write.c:954
>>  SYSC_writev fs/read_write.c:1027 [inline]
>>  SyS_writev+0x27/0x30 fs/read_write.c:1024
>>  entry_SYSCALL_64_fastpath+0x1f/0xc2
>> RIP: 0033:0x4458d9
>> RSP: 002b:00007f31fcf33b58 EFLAGS: 00000292 ORIG_RAX: 0000000000000014
>> RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 00000000004458d9
>> RDX: 0000000000000001 RSI: 00000000207cd000 RDI: 0000000000000005
>> RBP: 00000000006e30c0 R08: 0000000000000000 R09: 0000000000000000
>> R10: 0000000000000000 R11: 0000000000000292 R12: 0000000000708000
>> R13: 0000000020fad000 R14: 0000000000001000 R15: 0000000000000003
>>
>>
>>
>> ------------[ cut here ]------------
>> WARNING: CPU: 2 PID: 31175 at net/ipv6/ip6_fib.c:158
>> rt6_rcu_free+0x61/0x70 net/ipv6/ip6_fib.c:158
>> Kernel panic - not syncing: panic_on_warn set ...
>>
>> CPU: 2 PID: 31175 Comm: syz-executor1 Not tainted 4.11.0-rc1+ #310
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>> Call Trace:
>>  __dump_stack lib/dump_stack.c:16 [inline]
>>  dump_stack+0x2fb/0x3fd lib/dump_stack.c:52
>>  panic+0x20f/0x426 kernel/panic.c:180
>>  __warn+0x1c4/0x1e0 kernel/panic.c:541
>>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:584
>>  rt6_rcu_free+0x61/0x70 net/ipv6/ip6_fib.c:158
>>  rt6_release+0x1ee/0x290 net/ipv6/ip6_fib.c:189
>>  fib6_add_rt2node net/ipv6/ip6_fib.c:922 [inline]
>>  fib6_add+0x1d51/0x3290 net/ipv6/ip6_fib.c:1081
>> kvm_vm_ioctl_deassign_device: device hasn't been assigned before, so
>> cannot be deassigned
>>  __ip6_ins_rt+0x60/0x80 net/ipv6/route.c:948
>>  ip6_route_add+0x1a7/0x310 net/ipv6/route.c:2130
>>  inet6_rtm_newroute+0x191/0x1b0 net/ipv6/route.c:3294
>>  rtnetlink_rcv_msg+0x609/0x860 net/core/rtnetlink.c:4104
>>  netlink_rcv_skb+0x2ab/0x390 net/netlink/af_netlink.c:2298
>>  rtnetlink_rcv+0x2a/0x40 net/core/rtnetlink.c:4110
>>  netlink_unicast_kernel net/netlink/af_netlink.c:1231 [inline]
>>  netlink_unicast+0x525/0x730 net/netlink/af_netlink.c:1257
>>  netlink_sendmsg+0xab3/0xe70 net/netlink/af_netlink.c:1803
>>  sock_sendmsg_nosec net/socket.c:633 [inline]
>>  sock_sendmsg+0xca/0x110 net/socket.c:643
>>  sock_write_iter+0x326/0x600 net/socket.c:846
>>  call_write_iter include/linux/fs.h:1733 [inline]
>>  do_iter_readv_writev fs/read_write.c:696 [inline]
>>  __do_readv_writev+0xbbc/0x10a0 fs/read_write.c:862
>>  do_readv_writev+0x13f/0x200 fs/read_write.c:894
>>  vfs_writev+0x87/0xc0 fs/read_write.c:921
>>  do_writev+0x110/0x2c0 fs/read_write.c:954
>>  SYSC_writev fs/read_write.c:1027 [inline]
>>  SyS_writev+0x27/0x30 fs/read_write.c:1024
>>  entry_SYSCALL_64_fastpath+0x1f/0xc2
>> RIP: 0033:0x4458d9
>> RSP: 002b:00007f1639006b58 EFLAGS: 00000292 ORIG_RAX: 0000000000000014
>> RAX: ffffffffffffffda RBX: 0000000000000019 RCX: 00000000004458d9
>> RDX: 0000000000000001 RSI: 00000000207cd000 RDI: 0000000000000019
>> RBP: 00000000006e30c0 R08: 0000000000000000 R09: 0000000000000000
>> R10: 0000000000000000 R11: 0000000000000292 R12: 0000000000708000
>> R13: 0000000000000010 R14: 0000000000000003 R15: 0000000000000000
>>
>
> --
> You received this message because you are subscribed to the Google Groups "syzkaller" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to syzkaller+unsubscribe@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | linux.kernel

csiph-web