Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1570722 > unrolled thread

net/tcp: warning in tcp_try_coalesce/skb_try_coalesce

Started byAndrey Konovalov <andreyknvl@google.com>
First post2017-01-31 14:20 +0100
Last post2017-02-02 15:10 +0100
Articles 7 — 4 participants

Back to article view | Back to linux.kernel

Contents

  net/tcp: warning in tcp_try_coalesce/skb_try_coalesce Andrey Konovalov <andreyknvl@google.com> - 2017-01-31 14:20 +0100
    Re: net/tcp: warning in tcp_try_coalesce/skb_try_coalesce Eric Dumazet <edumazet@google.com> - 2017-01-31 15:30 +0100
      Re: net/tcp: warning in tcp_try_coalesce/skb_try_coalesce Dmitry Vyukov <dvyukov@google.com> - 2017-02-02 10:10 +0100
        Re: net/tcp: warning in tcp_try_coalesce/skb_try_coalesce Eric Dumazet <eric.dumazet@gmail.com> - 2017-02-02 13:30 +0100
          Re: net/tcp: warning in tcp_try_coalesce/skb_try_coalesce Dmitry Vyukov <dvyukov@google.com> - 2017-02-02 14:00 +0100
            Re: net/tcp: warning in tcp_try_coalesce/skb_try_coalesce Eric Dumazet <eric.dumazet@gmail.com> - 2017-02-02 14:50 +0100
              Re: net/tcp: warning in tcp_try_coalesce/skb_try_coalesce Dmitry Vyukov <dvyukov@google.com> - 2017-02-02 15:10 +0100

#1570722 — net/tcp: warning in tcp_try_coalesce/skb_try_coalesce

FromAndrey Konovalov <andreyknvl@google.com>
Date2017-01-31 14:20 +0100
Subjectnet/tcp: warning in tcp_try_coalesce/skb_try_coalesce
Message-ID<t5GvM-19j-25@gated-at.bofh.it>
Hi,

I've got the following error report while running the syzkaller fuzzer.

On commit 566cf877a1fcb6d6dc0126b076aad062054c2637 (4.10-rc6).

The fuzzer hits this issue quite often, but I don't have a working reproducer.

WARNING: CPU: 3 PID: 7091 at net/core/skbuff.c:4331
skb_try_coalesce+0x14b1/0x1f80 net/core/skbuff.c:4331
Kernel panic - not syncing: panic_on_warn set ...

CPU: 3 PID: 7091 Comm: syz-executor4 Not tainted 4.10.0-rc6+ #116
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:15 [inline]
 dump_stack+0x292/0x398 lib/dump_stack.c:51
 panic+0x1cb/0x3a9 kernel/panic.c:179
 __warn+0x1c4/0x1e0 kernel/panic.c:539
 warn_slowpath_null+0x2c/0x40 kernel/panic.c:582
 skb_try_coalesce+0x14b1/0x1f80 net/core/skbuff.c:4331
 tcp_try_coalesce+0x178/0x560 net/ipv4/tcp_input.c:4337
 tcp_queue_rcv+0x13c/0x7a0 net/ipv4/tcp_input.c:4572
 tcp_data_queue+0x97e/0x46b0 net/ipv4/tcp_input.c:4681
 tcp_rcv_state_process+0xd92/0x42d0 net/ipv4/tcp_input.c:6130
 tcp_v4_do_rcv+0x571/0x940 net/ipv4/tcp_ipv4.c:1426
 sk_backlog_rcv include/net/sock.h:893 [inline]
 __release_sock+0x127/0x3a0 net/core/sock.c:2053
 release_sock+0xa5/0x2b0 net/core/sock.c:2540
 inet_shutdown+0x1a8/0x350 net/ipv4/af_inet.c:833
 SYSC_shutdown net/socket.c:1840 [inline]
 SyS_shutdown+0x253/0x2f0 net/socket.c:1831
 entry_SYSCALL_64_fastpath+0x1f/0xc2
RIP: 0033:0x4457d9
RSP: 002b:00007f73b41a5b58 EFLAGS: 00000282 ORIG_RAX: 0000000000000030
RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 00000000004457d9
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000005
RBP: 00000000006e2170 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000282 R12: 0000000000708000
R13: 0000000000000005 R14: 0000000000000029 R15: 000000000000003f
Dumping ftrace buffer:
   (ftrace buffer empty)
Kernel Offset: disabled
Rebooting in 86400 seconds..

[toc] | [next] | [standalone]

#1570785

FromEric Dumazet <edumazet@google.com>
Date2017-01-31 15:30 +0100
Message-ID<t5HBw-1Ll-17@gated-at.bofh.it>
In reply to#1570722
On Tue, Jan 31, 2017 at 5:17 AM, Andrey Konovalov <andreyknvl@google.com> wrote:
> Hi,
>
> I've got the following error report while running the syzkaller fuzzer.
>
> On commit 566cf877a1fcb6d6dc0126b076aad062054c2637 (4.10-rc6).
>
> The fuzzer hits this issue quite often, but I don't have a working reproducer.
>
> WARNING: CPU: 3 PID: 7091 at net/core/skbuff.c:4331
> skb_try_coalesce+0x14b1/0x1f80 net/core/skbuff.c:4331
> Kernel panic - not syncing: panic_on_warn set ...
>
> CPU: 3 PID: 7091 Comm: syz-executor4 Not tainted 4.10.0-rc6+ #116
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:15 [inline]
>  dump_stack+0x292/0x398 lib/dump_stack.c:51
>  panic+0x1cb/0x3a9 kernel/panic.c:179
>  __warn+0x1c4/0x1e0 kernel/panic.c:539
>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:582
>  skb_try_coalesce+0x14b1/0x1f80 net/core/skbuff.c:4331
>  tcp_try_coalesce+0x178/0x560 net/ipv4/tcp_input.c:4337
>  tcp_queue_rcv+0x13c/0x7a0 net/ipv4/tcp_input.c:4572
>  tcp_data_queue+0x97e/0x46b0 net/ipv4/tcp_input.c:4681
>  tcp_rcv_state_process+0xd92/0x42d0 net/ipv4/tcp_input.c:6130
>  tcp_v4_do_rcv+0x571/0x940 net/ipv4/tcp_ipv4.c:1426
>  sk_backlog_rcv include/net/sock.h:893 [inline]
>  __release_sock+0x127/0x3a0 net/core/sock.c:2053
>  release_sock+0xa5/0x2b0 net/core/sock.c:2540
>  inet_shutdown+0x1a8/0x350 net/ipv4/af_inet.c:833
>  SYSC_shutdown net/socket.c:1840 [inline]
>  SyS_shutdown+0x253/0x2f0 net/socket.c:1831
>  entry_SYSCALL_64_fastpath+0x1f/0xc2
> RIP: 0033:0x4457d9
> RSP: 002b:00007f73b41a5b58 EFLAGS: 00000282 ORIG_RAX: 0000000000000030
> RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 00000000004457d9
> RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000005
> RBP: 00000000006e2170 R08: 0000000000000000 R09: 0000000000000000
> R10: 0000000000000000 R11: 0000000000000282 R12: 0000000000708000
> R13: 0000000000000005 R14: 0000000000000029 R15: 000000000000003f
> Dumping ftrace buffer:
>    (ftrace buffer empty)
> Kernel Offset: disabled
> Rebooting in 86400 seconds..

Have you tried the patch we added recently into net-next ?

https://git.kernel.org/cgit/linux/kernel/git/davem/net-next.git/commit/?id=158f323b9868b59967ad96957c4ca388161be321

[toc] | [prev] | [next] | [standalone]

#1572203

FromDmitry Vyukov <dvyukov@google.com>
Date2017-02-02 10:10 +0100
Message-ID<t6lyW-1O8-13@gated-at.bofh.it>
In reply to#1570785
On Tue, Jan 31, 2017 at 3:27 PM, 'Eric Dumazet' via syzkaller
<syzkaller@googlegroups.com> wrote:
> On Tue, Jan 31, 2017 at 5:17 AM, Andrey Konovalov <andreyknvl@google.com> wrote:
>> Hi,
>>
>> I've got the following error report while running the syzkaller fuzzer.
>>
>> On commit 566cf877a1fcb6d6dc0126b076aad062054c2637 (4.10-rc6).
>>
>> The fuzzer hits this issue quite often, but I don't have a working reproducer.
>>
>> WARNING: CPU: 3 PID: 7091 at net/core/skbuff.c:4331
>> skb_try_coalesce+0x14b1/0x1f80 net/core/skbuff.c:4331
>> Kernel panic - not syncing: panic_on_warn set ...
...
> Have you tried the patch we added recently into net-next ?
>
> https://git.kernel.org/cgit/linux/kernel/git/davem/net-next.git/commit/?id=158f323b9868b59967ad96957c4ca388161be321


I've tried this patch, and it does _not_ help.
I am currently on a2ca3d617944417e9dd5f09fc8a4549cda115f4f, and I
applied the patch on top (had to do a little merging in af_netlink.c):
https://gist.githubusercontent.com/dvyukov/b4fd206f7dde763b3113a4543c57e7bb/raw/784c5b1bcb06f2a22c55eb4c4f8f83d627cd1ab9/gistfile1.txt
Still getting the same warning.

But now I have a repro:
https://gist.githubusercontent.com/dvyukov/88fdb6d93507b36f81377191b0694f9b/raw/81cc11e8b78b6bbc5cffc79b442ce998c49af226/gistfile1.txt


WARNING: CPU: 2 PID: 2958 at net/core/skbuff.c:4339
skb_try_coalesce+0x154c/0x2020 net/core/skbuff.c:4339
Kernel panic - not syncing: panic_on_warn set ...
CPU: 2 PID: 2958 Comm: a.out Not tainted 4.10.0-rc6+ #206
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:15 [inline]
 dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
 panic+0x1fb/0x412 kernel/panic.c:179
 __warn+0x1c4/0x1e0 kernel/panic.c:539
 warn_slowpath_null+0x2c/0x40 kernel/panic.c:582
 skb_try_coalesce+0x154c/0x2020 net/core/skbuff.c:4339
 tcp_try_coalesce+0x16f/0x560 net/ipv4/tcp_input.c:4337
 tcp_queue_rcv+0x12d/0x790 net/ipv4/tcp_input.c:4572
 tcp_data_queue+0x96f/0x46b0 net/ipv4/tcp_input.c:4681
 tcp_rcv_state_process+0xd92/0x42c0 net/ipv4/tcp_input.c:6130
 tcp_v4_do_rcv+0x56b/0x940 net/ipv4/tcp_ipv4.c:1426
 sk_backlog_rcv include/net/sock.h:893 [inline]
 __release_sock+0x126/0x3a0 net/core/sock.c:2053
 release_sock+0xa5/0x2b0 net/core/sock.c:2540
 inet_shutdown+0x1a8/0x350 net/ipv4/af_inet.c:833
 SYSC_shutdown net/socket.c:1840 [inline]
 SyS_shutdown+0x23d/0x2d0 net/socket.c:1831
 entry_SYSCALL_64_fastpath+0x1f/0xc2
RIP: 0033:0x443b39
RSP: 002b:00007ffcfd0814f8 EFLAGS: 00000206 ORIG_RAX: 0000000000000030
RAX: ffffffffffffffda RBX: 00000000004031d0 RCX: 0000000000443b39
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000003
RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000206 R12: 00000000004002b0
R13: 00000000004031d0 R14: 0000000000403260 R15: 0000000000000000
Kernel Offset: disabled
Rebooting in 86400 seconds..

[toc] | [prev] | [next] | [standalone]

#1572323

FromEric Dumazet <eric.dumazet@gmail.com>
Date2017-02-02 13:30 +0100
Message-ID<t6oGu-3Tn-27@gated-at.bofh.it>
In reply to#1572203
On Thu, 2017-02-02 at 10:05 +0100, Dmitry Vyukov wrote:
> On Tue, Jan 31, 2017 at 3:27 PM, 'Eric Dumazet' via syzkaller
> <syzkaller@googlegroups.com> wrote:
> > On Tue, Jan 31, 2017 at 5:17 AM, Andrey Konovalov <andreyknvl@google.com> wrote:
> >> Hi,
> >>
> >> I've got the following error report while running the syzkaller fuzzer.
> >>
> >> On commit 566cf877a1fcb6d6dc0126b076aad062054c2637 (4.10-rc6).
> >>
> >> The fuzzer hits this issue quite often, but I don't have a working reproducer.
> >>
> >> WARNING: CPU: 3 PID: 7091 at net/core/skbuff.c:4331
> >> skb_try_coalesce+0x14b1/0x1f80 net/core/skbuff.c:4331
> >> Kernel panic - not syncing: panic_on_warn set ...
> ...
> > Have you tried the patch we added recently into net-next ?
> >
> > https://git.kernel.org/cgit/linux/kernel/git/davem/net-next.git/commit/?id=158f323b9868b59967ad96957c4ca388161be321
> 
> 
> I've tried this patch, and it does _not_ help.
> I am currently on a2ca3d617944417e9dd5f09fc8a4549cda115f4f, and I
> applied the patch on top (had to do a little merging in af_netlink.c):
> https://gist.githubusercontent.com/dvyukov/b4fd206f7dde763b3113a4543c57e7bb/raw/784c5b1bcb06f2a22c55eb4c4f8f83d627cd1ab9/gistfile1.txt
> Still getting the same warning.
> 
> But now I have a repro:
> https://gist.githubusercontent.com/dvyukov/88fdb6d93507b36f81377191b0694f9b/raw/81cc11e8b78b6bbc5cffc79b442ce998c49af226/gistfile1.txt
> 

Interesting. This repro does not trigger the bug on net-next.

[toc] | [prev] | [next] | [standalone]

#1572334

FromDmitry Vyukov <dvyukov@google.com>
Date2017-02-02 14:00 +0100
Message-ID<t6p9w-441-25@gated-at.bofh.it>
In reply to#1572323
On Thu, Feb 2, 2017 at 1:20 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> On Thu, 2017-02-02 at 10:05 +0100, Dmitry Vyukov wrote:
>> On Tue, Jan 31, 2017 at 3:27 PM, 'Eric Dumazet' via syzkaller
>> <syzkaller@googlegroups.com> wrote:
>> > On Tue, Jan 31, 2017 at 5:17 AM, Andrey Konovalov <andreyknvl@google.com> wrote:
>> >> Hi,
>> >>
>> >> I've got the following error report while running the syzkaller fuzzer.
>> >>
>> >> On commit 566cf877a1fcb6d6dc0126b076aad062054c2637 (4.10-rc6).
>> >>
>> >> The fuzzer hits this issue quite often, but I don't have a working reproducer.
>> >>
>> >> WARNING: CPU: 3 PID: 7091 at net/core/skbuff.c:4331
>> >> skb_try_coalesce+0x14b1/0x1f80 net/core/skbuff.c:4331
>> >> Kernel panic - not syncing: panic_on_warn set ...
>> ...
>> > Have you tried the patch we added recently into net-next ?
>> >
>> > https://git.kernel.org/cgit/linux/kernel/git/davem/net-next.git/commit/?id=158f323b9868b59967ad96957c4ca388161be321
>>
>>
>> I've tried this patch, and it does _not_ help.
>> I am currently on a2ca3d617944417e9dd5f09fc8a4549cda115f4f, and I
>> applied the patch on top (had to do a little merging in af_netlink.c):
>> https://gist.githubusercontent.com/dvyukov/b4fd206f7dde763b3113a4543c57e7bb/raw/784c5b1bcb06f2a22c55eb4c4f8f83d627cd1ab9/gistfile1.txt
>> Still getting the same warning.
>>
>> But now I have a repro:
>> https://gist.githubusercontent.com/dvyukov/88fdb6d93507b36f81377191b0694f9b/raw/81cc11e8b78b6bbc5cffc79b442ce998c49af226/gistfile1.txt
>>
>
> Interesting. This repro does not trigger the bug on net-next.


Right, I can confirm that this is not happening on net-next tip
(62e13097c46c69dbd7544ab2cd585ccf48f360a4).
I reproduced it on a random month old commit on net-next, though
(23a8ed4a624324dc696c328f09bd502c4a3816f0). Let's consider that this
is accidentally fixed by something in that range.

[toc] | [prev] | [next] | [standalone]

#1572373

FromEric Dumazet <eric.dumazet@gmail.com>
Date2017-02-02 14:50 +0100
Message-ID<t6pVU-4Ix-23@gated-at.bofh.it>
In reply to#1572334
On Thu, 2017-02-02 at 13:59 +0100, Dmitry Vyukov wrote:

> 
> Right, I can confirm that this is not happening on net-next tip
> (62e13097c46c69dbd7544ab2cd585ccf48f360a4).
> I reproduced it on a random month old commit on net-next, though
> (23a8ed4a624324dc696c328f09bd502c4a3816f0). Let's consider that this
> is accidentally fixed by something in that range.

This also might depend on the .config 

I tried 23a8ed4a624324dc696c328f09bd502c4a3816f0 and could not trigger
the warning .

Thanks.

[toc] | [prev] | [next] | [standalone]

#1572378

FromDmitry Vyukov <dvyukov@google.com>
Date2017-02-02 15:10 +0100
Message-ID<t6qff-54C-11@gated-at.bofh.it>
In reply to#1572373
On Thu, Feb 2, 2017 at 2:56 PM, Dmitry Vyukov <dvyukov@google.com> wrote:
> On Thu, Feb 2, 2017 at 2:40 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
>> On Thu, 2017-02-02 at 13:59 +0100, Dmitry Vyukov wrote:
>>
>>>
>>> Right, I can confirm that this is not happening on net-next tip
>>> (62e13097c46c69dbd7544ab2cd585ccf48f360a4).
>>> I reproduced it on a random month old commit on net-next, though
>>> (23a8ed4a624324dc696c328f09bd502c4a3816f0). Let's consider that this
>>> is accidentally fixed by something in that range.
>>
>> This also might depend on the .config
>>
>> I tried 23a8ed4a624324dc696c328f09bd502c4a3816f0 and could not trigger
>> the warning .
>
>
> I've attached my config.
> But I've tried with the same config on both revisions.
>
> Well, there is also a chance that the bug is not fixed, but requires
> slightly different reproducer after some changes.

Hummm....

My config includes KASAN+KCOV and I use gcc version 7.0.0 20161208
(experimental) (GCC).
I've now tried to turn off KASAN+KCOV and compiled with my stock
4.8.4, and this did _not_ trigger the warning on
23a8ed4a624324dc696c328f09bd502c4a3816f0.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web