Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1583935 > unrolled thread

Re: net: SOFTIRQ-safe -> SOFTIRQ-unsafe lock order detected in skb_array_produce

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: net: SOFTIRQ-safe -> SOFTIRQ-unsafe lock order detected in skb_array_produce Dmitry Vyukov <dvyukov@google.com> - 2017-02-18 18:30 +0100
    Re: net: SOFTIRQ-safe -> SOFTIRQ-unsafe lock order detected in skb_array_produce Dmitry Vyukov <dvyukov@google.com> - 2017-02-18 18:40 +0100
    Re: net: SOFTIRQ-safe -> SOFTIRQ-unsafe lock order detected in  skb_array_produce "Michael S. Tsirkin" <mst@redhat.com> - 2017-02-19 06:20 +0100

#1583935 — Re: net: SOFTIRQ-safe -> SOFTIRQ-unsafe lock order detected in skb_array_produce

On Fri, Feb 10, 2017 at 6:17 AM, Jason Wang <jasowang@redhat.com> wrote:
>
>
> On 2017年02月10日 02:10, Michael S. Tsirkin wrote:
>>
>> On Thu, Feb 09, 2017 at 05:02:31AM -0500, Jason Wang wrote:
>>>
>>> ----- Original Message -----
>>>>
>>>> Hello,
>>>>
>>>> I've got the following report while running syzkaller fuzzer on mmotm
>>>> (git://git.kernel.org/pub/scm/linux/kernel/git/mhocko/mm.git)
>>>> remotes/mmotm/auto-latest ee4ba7533626ba7bf2f8b992266467ac9fdc045e:
>>>>
>>> [...]
>>>
>>>> other info that might help us debug this:
>>>>
>>>>   Possible interrupt unsafe locking scenario:
>>>>
>>>>         CPU0                    CPU1
>>>>         ----                    ----
>>>>    lock(&(&r->consumer_lock)->rlock);
>>>>                                 local_irq_disable();
>>>>                                 lock(&(&r->producer_lock)->rlock);
>>>>                                 lock(&(&r->consumer_lock)->rlock);
>>>>    <Interrupt>
>>>>      lock(&(&r->producer_lock)->rlock);
>>>>
>>> Thanks a lot for the testing.
>>>
>>> Looks like we could address this by using skb_array_consume_bh() instead.
>>>
>>> Could you pls verify if the following patch works?
>>
>> I think we should use _bh for the produce call as well,
>> since resizing takes the producer lock.
>
> Looks not since irq was disabled during resizing?


Hello,

Is there a fix for this that we can pick up?
This killed 10'000 VMs on our testing infra over the last day. Still
happening on linux-next.

Thanks

[toc] | [next] | [standalone]

#1583938

On Sat, Feb 18, 2017 at 6:28 PM, Dmitry Vyukov <dvyukov@google.com> wrote:
> On Fri, Feb 10, 2017 at 6:17 AM, Jason Wang <jasowang@redhat.com> wrote:
>>
>>
>> On 2017年02月10日 02:10, Michael S. Tsirkin wrote:
>>>
>>> On Thu, Feb 09, 2017 at 05:02:31AM -0500, Jason Wang wrote:
>>>>
>>>> ----- Original Message -----
>>>>>
>>>>> Hello,
>>>>>
>>>>> I've got the following report while running syzkaller fuzzer on mmotm
>>>>> (git://git.kernel.org/pub/scm/linux/kernel/git/mhocko/mm.git)
>>>>> remotes/mmotm/auto-latest ee4ba7533626ba7bf2f8b992266467ac9fdc045e:
>>>>>
>>>> [...]
>>>>
>>>>> other info that might help us debug this:
>>>>>
>>>>>   Possible interrupt unsafe locking scenario:
>>>>>
>>>>>         CPU0                    CPU1
>>>>>         ----                    ----
>>>>>    lock(&(&r->consumer_lock)->rlock);
>>>>>                                 local_irq_disable();
>>>>>                                 lock(&(&r->producer_lock)->rlock);
>>>>>                                 lock(&(&r->consumer_lock)->rlock);
>>>>>    <Interrupt>
>>>>>      lock(&(&r->producer_lock)->rlock);
>>>>>
>>>> Thanks a lot for the testing.
>>>>
>>>> Looks like we could address this by using skb_array_consume_bh() instead.
>>>>
>>>> Could you pls verify if the following patch works?
>>>
>>> I think we should use _bh for the produce call as well,
>>> since resizing takes the producer lock.
>>
>> Looks not since irq was disabled during resizing?
>
>
> Hello,
>
> Is there a fix for this that we can pick up?
> This killed 10'000 VMs on our testing infra over the last day. Still
> happening on linux-next.


Ah, sorry, I see the patch above with skb_array_consume_bh. It's just
that it's not in linux-next. Will manually apply it now then.
Should we also do something with produce_skb?

[toc] | [prev] | [next] | [standalone]

#1584065 — Re: net: SOFTIRQ-safe -> SOFTIRQ-unsafe lock order detected in skb_array_produce

On Sat, Feb 18, 2017 at 06:28:39PM +0100, Dmitry Vyukov wrote:
> On Fri, Feb 10, 2017 at 6:17 AM, Jason Wang <jasowang@redhat.com> wrote:
> >
> >
> > On 2017年02月10日 02:10, Michael S. Tsirkin wrote:
> >>
> >> On Thu, Feb 09, 2017 at 05:02:31AM -0500, Jason Wang wrote:
> >>>
> >>> ----- Original Message -----
> >>>>
> >>>> Hello,
> >>>>
> >>>> I've got the following report while running syzkaller fuzzer on mmotm
> >>>> (git://git.kernel.org/pub/scm/linux/kernel/git/mhocko/mm.git)
> >>>> remotes/mmotm/auto-latest ee4ba7533626ba7bf2f8b992266467ac9fdc045e:
> >>>>
> >>> [...]
> >>>
> >>>> other info that might help us debug this:
> >>>>
> >>>>   Possible interrupt unsafe locking scenario:
> >>>>
> >>>>         CPU0                    CPU1
> >>>>         ----                    ----
> >>>>    lock(&(&r->consumer_lock)->rlock);
> >>>>                                 local_irq_disable();
> >>>>                                 lock(&(&r->producer_lock)->rlock);
> >>>>                                 lock(&(&r->consumer_lock)->rlock);
> >>>>    <Interrupt>
> >>>>      lock(&(&r->producer_lock)->rlock);
> >>>>
> >>> Thanks a lot for the testing.
> >>>
> >>> Looks like we could address this by using skb_array_consume_bh() instead.
> >>>
> >>> Could you pls verify if the following patch works?
> >>
> >> I think we should use _bh for the produce call as well,
> >> since resizing takes the producer lock.
> >
> > Looks not since irq was disabled during resizing?
> 
> 
> Hello,
> 
> Is there a fix for this that we can pick up?
> This killed 10'000 VMs on our testing infra over the last day. Still
> happening on linux-next.
> 
> Thanks

I posted a fix.  ptr_ring: fix race conditions when resizing
Just reposted.  I'll push into linux-next ASAP.

-- 
MST

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web