Re: net: deadlock between ip_expire/sch_direct_xmit

Path	csiph.com!aioe.org!bofh.it!news.nic.it!robomod
From	Cong Wang <xiyou.wangcong@gmail.com>
Newsgroups	linux.kernel
Subject	Re: net: deadlock between ip_expire/sch_direct_xmit
Date	Tue, 14 Mar 2017 17:50:01 +0100
Message-ID	<tkXO1-2Py-13@gated-at.bofh.it> (permalink)
References	<tkUQb-Mo-53@gated-at.bofh.it> <tkVVU-1uM-17@gated-at.bofh.it> <tkW5z-1Ad-3@gated-at.bofh.it> <tkW5z-1Ad-1@gated-at.bofh.it>
X-Original-To	Eric Dumazet <edumazet@google.com>
Dkim-Signature	v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=OxmLmG5tiKWHt+Y/MqeRSYofs21IIhhaxkEMT7QCIFc=; b=Rif3mTlS5Zwqet+4oPA5o9gogkief87R3oJ8P3uyGPQ3//M4tjRrlyVdckD+7id/j1 YVLNyx3TQLNWxYxH/RB4W5YgptM7Q5vNfQ4VlfT+93kkRfnbJkMZ2wR6x/NCbeyzX5DR k/7SG8t67/8qzgHxIO1ixjuvXkinEtu/cxjd0XY1vLDMZvOPV2AhCQ00NL8/cPBWZ3xP RPmaQHUNR7BBT6gPlKN2K8CT9+xf+VupADowus4deYAFzn8ch+/e85BYEQK9yaJGAp/b F5+pmi7eJIvzhPav4qvjes7a6NINE0vseSNqBl5YYpEKIDUN70Y82ivuNA5hhHFRdIzO Rlpg==
X-Google-Dkim-Signature	v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=OxmLmG5tiKWHt+Y/MqeRSYofs21IIhhaxkEMT7QCIFc=; b=F68gAutStlARCCnn5sk4DoNU9dULwcxVJCBET0W7cJWDayK2Cp7WB/Scy8mjJZoZUO 1wiXpKPvJgbSmMd3eZSZICLCCImAkJdRAZmnnKEikF72lU0hyTVYVL+mwDlK8qlW6I45 e+Y55CiCDplO5BQnE0Gq8W67kPfe/nK0zMNnSX5gJl/qOGhTpasybFlLpmVwj2sP1N84 BNAugSHKNgaikFSYPotoUDjHnyYBBV43KCfXUek2XoR1G86sykgExGPcR+MoLK3P710A fCpiFiohD0p45IxpjxvfidJZJlWuIIbHrMhTRP83nuFUf8JA9t0RU42/qbvjWdMoZPEs /HFQ==
X-Gm-Message-State	AFeK/H2qc2x1HE6yMwB+vWR8H1D3YthZxlZSyPOu50CmCl7vRul/HyIzX5+ZGv/JQhEwR4gjIvWr6twCEcQEsQ==
X-Received	by 10.28.131.77 with SMTP id f74mr15404140wmd.109.1489509702832; Tue, 14 Mar 2017 09:41:42 -0700 (PDT)
MIME-Version	1.0
Content-Type	text/plain; charset=UTF-8
Sender	robomod@news.nic.it
List-ID	<linux-kernel.vger.kernel.org>
X-Mailing-List	linux-kernel@vger.kernel.org
Approved	robomod@news.nic.it
Lines	31
Organization	linux.* mail to news gateway
X-Original-Cc	Dmitry Vyukov <dvyukov@google.com>, Eric Dumazet <eric.dumazet@gmail.com>, David Miller <davem@davemloft.net>, Alexey Kuznetsov <kuznet@ms2.inr.ac.ru>, James Morris <jmorris@namei.org>, Hideaki YOSHIFUJI <yoshfuji@linux-ipv6.org>, Patrick McHardy <kaber@trash.net>, netdev <netdev@vger.kernel.org>, LKML <linux-kernel@vger.kernel.org>, Jamal Hadi Salim <jhs@mojatatu.com>, syzkaller <syzkaller@googlegroups.com>
X-Original-Date	Tue, 14 Mar 2017 09:41:22 -0700
X-Original-Message-ID	<CAM_iQpV9iyOHoYUUO=wwHWz0GpUoQzwX+f3DSxwJo54eCcqH2g@mail.gmail.com>
X-Original-References	<CACT4Y+ZrHr0Cqw5RPeZ6QW16auOPyKSCOea6AciHBswAT3t14Q@mail.gmail.com> <1489502504.28631.115.camel@edumazet-glaptop3.roam.corp.google.com> <CACT4Y+bLJtO55iP5uNPZV+_B+H5-Z2gOiF26sXDmWk9L0rESsw@mail.gmail.com> <CANn89iLO1neA3-4ipYr==n_3iDXDXgY0MCkPkp=cEf8n4w6i=g@mail.gmail.com>
X-Original-Sender	linux-kernel-owner@vger.kernel.org
Xref	csiph.com linux.kernel:1600687

Show key headers only | View raw

On Tue, Mar 14, 2017 at 7:56 AM, Eric Dumazet <edumazet@google.com> wrote:
> On Tue, Mar 14, 2017 at 7:46 AM, Dmitry Vyukov <dvyukov@google.com> wrote:
>
>> I am confused. Lockdep has observed both of these stacks:
>>
>>        CPU0                    CPU1
>>        ----                    ----
>>   lock(&(&q->lock)->rlock);
>>                                lock(_xmit_ETHER#2);
>>                                lock(&(&q->lock)->rlock);
>>   lock(_xmit_ETHER#2);
>>
>>
>> So it somehow happened. Or what do you mean?
>>
>
> Lockdep said " possible circular locking dependency detected " .
> It is not an actual deadlock, but lockdep machinery firing.
>
> For a dead lock to happen, this would require that he ICMP message
> sent by ip_expire() is itself fragmented and reassembled.
> This cannot be, because ICMP messages are not candidates for
> fragmentation, but lockdep can not know that of course...

It doesn't have to be ICMP, as long as get the same hash for
the inet_frag_queue, we will need to take the same lock and
deadlock will happen.

        hash = ipqhashfn(iph->id, iph->saddr, iph->daddr, iph->protocol);

So it is really up to this hash function.

Thread

net: deadlock between ip_expire/sch_direct_xmit Dmitry Vyukov <dvyukov@google.com> - 2017-03-14 14:40 +0100
  Re: net: deadlock between ip_expire/sch_direct_xmit Eric Dumazet <eric.dumazet@gmail.com> - 2017-03-14 15:50 +0100
    Re: net: deadlock between ip_expire/sch_direct_xmit Eric Dumazet <edumazet@google.com> - 2017-03-14 16:00 +0100
      Re: net: deadlock between ip_expire/sch_direct_xmit Dmitry Vyukov <dvyukov@google.com> - 2017-03-14 16:00 +0100
      Re: net: deadlock between ip_expire/sch_direct_xmit Cong Wang <xiyou.wangcong@gmail.com> - 2017-03-14 17:50 +0100
        Re: net: deadlock between ip_expire/sch_direct_xmit Dmitry Vyukov <dvyukov@google.com> - 2017-03-20 11:10 +0100
          Re: net: deadlock between ip_expire/sch_direct_xmit Eric Dumazet <eric.dumazet@gmail.com> - 2017-03-20 13:50 +0100
    Re: net: deadlock between ip_expire/sch_direct_xmit Dmitry Vyukov <dvyukov@google.com> - 2017-03-14 16:00 +0100
    Re: net: deadlock between ip_expire/sch_direct_xmit Paolo Abeni <pabeni@redhat.com> - 2017-03-14 16:10 +0100
      Re: net: deadlock between ip_expire/sch_direct_xmit Eric Dumazet <edumazet@google.com> - 2017-03-14 16:20 +0100
        Re: net: deadlock between ip_expire/sch_direct_xmit Paolo Abeni <pabeni@redhat.com> - 2017-03-14 16:40 +0100

csiph-web