Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1271287 > unrolled thread

Asterisk deadlocks since Kernel 4.1

Back to article view | Back to linux.kernel

Contents

  Asterisk deadlocks since Kernel 4.1 Stefan Priebe - Profihost AG <s.priebe@profihost.ag> - 2015-11-17 15:50 +0100
    Re: Asterisk deadlocks since Kernel 4.1 Thomas Gleixner <tglx@linutronix.de> - 2015-11-17 20:20 +0100
      Re: Asterisk deadlocks since Kernel 4.1 Stefan Priebe <s.priebe@profihost.ag> - 2015-11-17 20:30 +0100
        Re: Asterisk deadlocks since Kernel 4.1 Thomas Gleixner <tglx@linutronix.de> - 2015-11-17 20:50 +0100
          Re: Asterisk deadlocks since Kernel 4.1 Stefan Priebe <s.priebe@profihost.ag> - 2015-11-18 21:30 +0100
            Re: Asterisk deadlocks since Kernel 4.1 Hannes Frederic Sowa <hannes@stressinduktion.org> - 2015-11-18 22:10 +0100
              Re: Asterisk deadlocks since Kernel 4.1 Stefan Priebe <s.priebe@profihost.ag> - 2015-11-18 22:30 +0100
                Re: Asterisk deadlocks since Kernel 4.1 Hannes Frederic Sowa <hannes@stressinduktion.org> - 2015-11-18 22:30 +0100
                  Re: Asterisk deadlocks since Kernel 4.1 Stefan Priebe - Profihost AG <s.priebe@profihost.ag> - 2015-11-19 10:40 +0100
            Re: Asterisk deadlocks since Kernel 4.1 Florian Weimer <fweimer@redhat.com> - 2015-11-18 22:20 +0100
              Re: Asterisk deadlocks since Kernel 4.1 Stefan Priebe <s.priebe@profihost.ag> - 2015-11-18 22:30 +0100
                Re: Asterisk deadlocks since Kernel 4.1 Florian Weimer <fweimer@redhat.com> - 2015-11-19 10:40 +0100
              Re: Asterisk deadlocks since Kernel 4.1 Stefan Priebe <s.priebe@profihost.ag> - 2015-11-18 22:40 +0100
                Re: Asterisk deadlocks since Kernel 4.1 Hannes Frederic Sowa <hannes@stressinduktion.org> - 2015-11-18 22:50 +0100
                  Re: Asterisk deadlocks since Kernel 4.1 Stefan Priebe <s.priebe@profihost.ag> - 2015-11-18 22:50 +0100
                    Re: Asterisk deadlocks since Kernel 4.1 Hannes Frederic Sowa <hannes@stressinduktion.org> - 2015-11-18 23:00 +0100
                Re: Asterisk deadlocks since Kernel 4.1 Stefan Priebe - Profihost AG <s.priebe@profihost.ag> - 2015-11-19 10:50 +0100
                  Re: Asterisk deadlocks since Kernel 4.1 Stefan Priebe - Profihost AG <s.priebe@profihost.ag> - 2015-11-19 11:00 +0100
                    Re: Asterisk deadlocks since Kernel 4.1 Hannes Frederic Sowa <hannes@stressinduktion.org> - 2015-11-19 12:50 +0100
                      Re: Asterisk deadlocks since Kernel 4.1 Stefan Priebe - Profihost AG <s.priebe@profihost.ag> - 2015-11-19 12:50 +0100
                        Re: Asterisk deadlocks since Kernel 4.1 Hannes Frederic Sowa <hannes@stressinduktion.org> - 2015-11-19 13:50 +0100
                          Re: Asterisk deadlocks since Kernel 4.1 Stefan Priebe - Profihost AG <s.priebe@profihost.ag> - 2015-11-19 13:50 +0100
                            Re: Asterisk deadlocks since Kernel 4.1 Florian Weimer <fweimer@redhat.com> - 2015-11-19 14:20 +0100
                              Re: Asterisk deadlocks since Kernel 4.1 Stefan Priebe <s.priebe@profihost.ag> - 2015-11-19 21:00 +0100
                                Re: Asterisk deadlocks since Kernel 4.1 Stefan Priebe - Profihost AG <s.priebe@profihost.ag> - 2015-11-23 13:50 +0100
                                  Re: Asterisk deadlocks since Kernel 4.1 Hannes Frederic Sowa <hannes@stressinduktion.org> - 2015-11-23 14:00 +0100
                                    Re: Asterisk deadlocks since Kernel 4.1 Stefan Priebe - Profihost AG <s.priebe@profihost.ag> - 2015-11-24 14:40 +0100
                Re: Asterisk deadlocks since Kernel 4.1 Florian Weimer <fweimer@redhat.com> - 2015-11-19 10:50 +0100

Page 2 of 2 — ← Prev page 1 [2]

#1273107

On Thu, Nov 19, 2015, at 12:43, Stefan Priebe - Profihost AG wrote:
> 
> Am 19.11.2015 um 12:41 schrieb Hannes Frederic Sowa:
> > On Thu, Nov 19, 2015, at 10:56, Stefan Priebe - Profihost AG wrote:
> >> OK it had a livelock again. It just took more time.
> >>
> >> So here is the data:
> > 
> > Thanks, I couldn't reproduce it so far with simple threaded resolver
> > loop on your kernel. :/
> > 
> > Your data is useless if you don't also provide the file descriptor which
> > you are blocking on right now. ;)
> > 
> > Thanks,
> > Hannes
> 
> ah sorry. So we need the gdb backtrace with the rcvmsg and then the fd
> list + netlink list?

Yes, albeit the probability we find something new is minimal. We already
know it blocks deep down in netlink code. I tried to reproduce it in
multithreaded environment with no results so far. Can you even try a
newer kernel?

Bye,
Hannes
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1273110

Am 19.11.2015 um 13:41 schrieb Hannes Frederic Sowa:
> On Thu, Nov 19, 2015, at 12:43, Stefan Priebe - Profihost AG wrote:
>>
>> Am 19.11.2015 um 12:41 schrieb Hannes Frederic Sowa:
>>> On Thu, Nov 19, 2015, at 10:56, Stefan Priebe - Profihost AG wrote:
>>>> OK it had a livelock again. It just took more time.
>>>>
>>>> So here is the data:
>>>
>>> Thanks, I couldn't reproduce it so far with simple threaded resolver
>>> loop on your kernel. :/
>>>
>>> Your data is useless if you don't also provide the file descriptor which
>>> you are blocking on right now. ;)
>>>
>>> Thanks,
>>> Hannes
>>
>> ah sorry. So we need the gdb backtrace with the rcvmsg and then the fd
>> list + netlink list?
> 
> Yes, albeit the probability we find something new is minimal. We already
> know it blocks deep down in netlink code. I tried to reproduce it in
> multithreaded environment with no results so far. Can you even try a
> newer kernel?

I can try Kernel 4.4-rc1 next week. Or something else?

Stefan

> 
> Bye,
> Hannes
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1273121

On 11/19/2015 01:46 PM, Stefan Priebe - Profihost AG wrote:

> I can try Kernel 4.4-rc1 next week. Or something else?

I found this bug report which indicates that 4.1.10 works:

  <https://issues.asterisk.org/jira/browse/ASTERISK-25251>

But in your original report, you said that 4.1.13 is broken.

This backtrace:

  <https://issues.asterisk.org/jira/secure/attachment/52702/bt.txt>

shows a lot of waiting on quite different netlink sockets.  So if this
is due to a race in Asterisk, it must have happened several times in a row.

Thanks,
Florian
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1273443

Am 19.11.2015 um 14:19 schrieb Florian Weimer:
> On 11/19/2015 01:46 PM, Stefan Priebe - Profihost AG wrote:
>
>> I can try Kernel 4.4-rc1 next week. Or something else?
>
> I found this bug report which indicates that 4.1.10 works:
>
>    <https://issues.asterisk.org/jira/browse/ASTERISK-25251>
>
> But in your original report, you said that 4.1.13 is broken.

That's correct i'm running 4.1.13.

> This backtrace:
>
>    <https://issues.asterisk.org/jira/secure/attachment/52702/bt.txt>
>
> shows a lot of waiting on quite different netlink sockets.  So if this
> is due to a race in Asterisk, it must have happened several times in a row.

Might be I've always only one waiting.

Stefan


> Thanks,
> Florian
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1275286

Am 19.11.2015 um 20:51 schrieb Stefan Priebe:
> 
> Am 19.11.2015 um 14:19 schrieb Florian Weimer:
>> On 11/19/2015 01:46 PM, Stefan Priebe - Profihost AG wrote:
>>
>>> I can try Kernel 4.4-rc1 next week. Or something else?
>>
>> I found this bug report which indicates that 4.1.10 works:
>>
>>    <https://issues.asterisk.org/jira/browse/ASTERISK-25251>
>>
>> But in your original report, you said that 4.1.13 is broken.
> 
> That's correct i'm running 4.1.13.
> 
>> This backtrace:
>>
>>    <https://issues.asterisk.org/jira/secure/attachment/52702/bt.txt>
>>
>> shows a lot of waiting on quite different netlink sockets.  So if this
>> is due to a race in Asterisk, it must have happened several times in a
>> row.

Kernel 4.4-rc2 works fine. How can we grab / get an idea which is
causing the isse in 4.1? It's an LTE kernel so it should be fixed!

Stefan
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1275298

On Mon, Nov 23, 2015, at 13:44, Stefan Priebe - Profihost AG wrote:
> Am 19.11.2015 um 20:51 schrieb Stefan Priebe:
> > 
> > Am 19.11.2015 um 14:19 schrieb Florian Weimer:
> >> On 11/19/2015 01:46 PM, Stefan Priebe - Profihost AG wrote:
> >>
> >>> I can try Kernel 4.4-rc1 next week. Or something else?
> >>
> >> I found this bug report which indicates that 4.1.10 works:
> >>
> >>    <https://issues.asterisk.org/jira/browse/ASTERISK-25251>
> >>
> >> But in your original report, you said that 4.1.13 is broken.
> > 
> > That's correct i'm running 4.1.13.
> > 
> >> This backtrace:
> >>
> >>    <https://issues.asterisk.org/jira/secure/attachment/52702/bt.txt>
> >>
> >> shows a lot of waiting on quite different netlink sockets.  So if this
> >> is due to a race in Asterisk, it must have happened several times in a
> >> row.
> 
> Kernel 4.4-rc2 works fine. How can we grab / get an idea which is
> causing the isse in 4.1? It's an LTE kernel so it should be fixed!

Thanks for testing. I was not able to reproduce it at all, with as much
parallelism and threads as possible on any kernel. Could you try to do a
git bisect?

Thanks,
Hannes
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1276452

Am 23.11.2015 um 13:57 schrieb Hannes Frederic Sowa:
> On Mon, Nov 23, 2015, at 13:44, Stefan Priebe - Profihost AG wrote:
>> Am 19.11.2015 um 20:51 schrieb Stefan Priebe:
>>>
>>> Am 19.11.2015 um 14:19 schrieb Florian Weimer:
>>>> On 11/19/2015 01:46 PM, Stefan Priebe - Profihost AG wrote:
>>>>
>>>>> I can try Kernel 4.4-rc1 next week. Or something else?
>>>>
>>>> I found this bug report which indicates that 4.1.10 works:
>>>>
>>>>    <https://issues.asterisk.org/jira/browse/ASTERISK-25251>
>>>>
>>>> But in your original report, you said that 4.1.13 is broken.
>>>
>>> That's correct i'm running 4.1.13.
>>>
>>>> This backtrace:
>>>>
>>>>    <https://issues.asterisk.org/jira/secure/attachment/52702/bt.txt>
>>>>
>>>> shows a lot of waiting on quite different netlink sockets.  So if this
>>>> is due to a race in Asterisk, it must have happened several times in a
>>>> row.
>>
>> Kernel 4.4-rc2 works fine. How can we grab / get an idea which is
>> causing the isse in 4.1? It's an LTE kernel so it should be fixed!
> 
> Thanks for testing. I was not able to reproduce it at all, with as much
> parallelism and threads as possible on any kernel. Could you try to do a
> git bisect?

OK i'll start digging around with major versions first to have a working
and a non working version.

I'll try 4.3 and 4.2 first to get an idea where it got fixed. Sadly i
can't bisect from 4.1.13 to 4.X so i have to verify first it also
happens with 4.1.

OK?

Stefan

> 
> Thanks,
> Hannes
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1272970

On 11/18/2015 10:36 PM, Stefan Priebe wrote:

>> please try to get a backtrace with debugging information.  It is likely
>> that this is the make_request/__check_pf functionality in glibc, but it
>> would be nice to get some certainty.
> 
> sorry here it is. What I'm wondering is why is there ipv6 stuff? I don't
> have ipv6 except for link local.

glibc needs to know if the system has global unicast addresses if it
receives AAAA records.

It's curious that net.ipv6.conf.all.disable_ipv6=1 makes the problem go
away.  Even with that setting, the kernel seems to send two Netlink
responses.  So either this is enough to narrow the window for the race
so that no longer triggers, or there is a genuine kernel issue with
supplying the requested IPv6 Netlink response.

> Could it be this one?
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=505105#c79

No, that's on the DNS/UDP side, not in the Netlink code.

Florian
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | linux.kernel

csiph-web