Re: [REGRESSION] tcp/ipv4: kernel panic because of (possible) division by zero

From	Yuchung Cheng <ycheng@google.com>
Newsgroups	linux.kernel
Subject	Re: [REGRESSION] tcp/ipv4: kernel panic because of (possible) division by zero
Date	2016-01-06 19:50 +0100
Message-ID	<qO1jH-4UD-9@gated-at.bofh.it> (permalink)
References	<qIfpn-2P6-1@gated-at.bofh.it> <qIkIp-6cV-3@gated-at.bofh.it> <qNZBf-3Kn-1@gated-at.bofh.it> <qO10m-4M5-5@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

On Wed, Jan 6, 2016 at 10:19 AM, Yuchung Cheng <ycheng@google.com> wrote:
> On Wed, Jan 6, 2016 at 8:50 AM, Oleksandr Natalenko
> <oleksandr@natalenko.name> wrote:
>>
>> Unfortunately, the patch didn't help -- I've got the same stacktrace with slightly different offset (+3) within the function.
>>
>> Now trying to get full stacktrace via netconsole. Need more time.
>>
>> Meanwhile, any other ideas on what went wrong?
>
> That's odd b/c the patch already checks and avoids div0. Can post me
> the stacktrace and kernel warnings if any ...
>
> One possibility is that tcp_cwnd_reduction() may set a cwnd of 0,
> which then gets used to start another recovery phase. This may or may
> not be the culprit of this div0 issue because I wasn't able to
> reproduce exactly your issue on our servers. But I will post the fix
> today and CC you.
Could you turn off ecn (sysctl net.ipv4.tcp_ecn=0) to see if this still happen?

>
>>
>>
>> On December 22, 2015 4:10:32 AM EET, Yuchung Cheng <ycheng@google.com> wrote:
>> >On Mon, Dec 21, 2015 at 12:25 PM, Oleksandr Natalenko
>> ><oleksandr@natalenko.name> wrote:
>> >> Commit 3759824da87b30ce7a35b4873b62b0ba38905ef5 (tcp: PRR uses CRB
>> >mode by
>> >> default and SS mode conditionally) introduced changes to
>> >net/ipv4/tcp_input.c
>> >> tcp_cwnd_reduction() that, possibly, cause division by zero, and
>> >therefore,
>> >> kernel panic in interrupt handler [1].
>> >>
>> >> Reverting 3759824da87b30ce7a35b4873b62b0ba38905ef5 seems to fix the
>> >issue.
>> >>
>> >> I'm able to reproduce the issue on 4.3.0–4.3.3 once per several day
>> >> (occasionally).
>> >>
>> >> What could be done to help in debugging this issue?
>> >Do you have ECN enabled (i.e. sysctl net.ipv4.tcp_ecn > 0)?
>> >
>> >If so I suspect an ACK carrying ECE during CA_Loss causes entering CWR
>> >state w/o calling tcp_init_cwnd_reduct() to set tp->prior_cwnd. Can
>> >you try this debug / quick-fix patch and send me the error message if
>> >any?
>> >
>> >
>> >>
>> >> Regards,
>> >>   Oleksandr.
>> >>
>> >> [1] http://i.piccy.info/
>> >>
>> >i9/6f5cb187c4ff282d189f78c63f95af43/1450729403/283985/951663/panic.jpg
>>
>> --
>> Sent from my Android device with K-9 Mail. Please excuse my brevity.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Thread

Re: [REGRESSION] tcp/ipv4: kernel panic because of (possible) division by zero Oleksandr Natalenko <oleksandr@natalenko.name> - 2016-01-06 18:00 +0100
  Re: [REGRESSION] tcp/ipv4: kernel panic because of (possible)  division by zero Yuchung Cheng <ycheng@google.com> - 2016-01-06 19:30 +0100
    Re: [REGRESSION] tcp/ipv4: kernel panic because of (possible) division by zero Oleksandr Natalenko <oleksandr@natalenko.name> - 2016-01-06 19:50 +0100
    Re: [REGRESSION] tcp/ipv4: kernel panic because of (possible)  division by zero Yuchung Cheng <ycheng@google.com> - 2016-01-06 19:50 +0100
      Re: [REGRESSION] tcp/ipv4: kernel panic because of (possible) division by zero Oleksandr Natalenko <oleksandr@natalenko.name> - 2016-01-09 18:40 +0100
      Re: [REGRESSION] tcp/ipv4: kernel panic because of (possible) division by zero Oleksandr Natalenko <oleksandr@natalenko.name> - 2016-01-10 11:30 +0100
        Re: [REGRESSION] tcp/ipv4: kernel panic because of (possible)  division by zero Neal Cardwell <ncardwell@google.com> - 2016-01-10 15:50 +0100
          Re: [REGRESSION] tcp/ipv4: kernel panic because of (possible)  division by zero Neal Cardwell <ncardwell@google.com> - 2016-01-10 16:00 +0100
            Re: [REGRESSION] tcp/ipv4: kernel panic because of (possible) division by zero Oleksandr Natalenko <oleksandr@natalenko.name> - 2016-01-10 16:00 +0100
          Re: [REGRESSION] tcp/ipv4: kernel panic because of (possible) division by zero Oleksandr Natalenko <oleksandr@natalenko.name> - 2016-01-10 16:00 +0100
            Re: [REGRESSION] tcp/ipv4: kernel panic because of (possible)  division by zero Neal Cardwell <ncardwell@google.com> - 2016-01-10 18:40 +0100
              Re: [REGRESSION] tcp/ipv4: kernel panic because of (possible) division by zero Oleksandr Natalenko <oleksandr@natalenko.name> - 2016-01-10 19:00 +0100
                Re: [REGRESSION] tcp/ipv4: kernel panic because of (possible)  division by zero Neal Cardwell <ncardwell@google.com> - 2016-01-10 19:10 +0100
              Re: [REGRESSION] tcp/ipv4: kernel panic because of (possible) division by zero Oleksandr Natalenko <oleksandr@natalenko.name> - 2016-01-10 23:00 +0100
                Re: [REGRESSION] tcp/ipv4: kernel panic because of (possible)  division by zero Neal Cardwell <ncardwell@google.com> - 2016-01-11 19:50 +0100
                Re: [REGRESSION] tcp/ipv4: kernel panic because of (possible) division by zero Oleksandr Natalenko <oleksandr@natalenko.name> - 2016-01-12 00:30 +0100

csiph-web