Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1525704

Re: Regression: Failed boots bisected to 4cd13c21b207 "softirq: Let ksoftirqd do its job"

Show all headers | View raw

Hi Thomas,

On Fri, Nov 18, 2016 at 01:40:43AM +0100, Thomas Gleixner wrote:
>Brian,
>
>On Thu, 17 Nov 2016, Brian Starkey wrote:
>> No joy with this patch :-(
>>
>> I had to add an ioaddr argument because apparently that macro depends
>> on local context (yuck...), but it doesn't help my issue.
>>
>> FWIW I don't see any timeouts, either with or without the patch.
>> (I don't know for sure, but I would guess that the model of the
>> network card doesn't model whatever stall that loop is checking for.
>> It probably just completes all MMU operations immediately)
>
>Is there a chance that you enable trace points at the kernel command line?
>
>  trace_event=sched_wakeup,sched_switch,irq_handler_entry,irq_handler_exit,softirq_raise,softirq_entry,softirq_exit
>
>should be enough for a start. All we need aside of that is a trigger to
>stop the trace so we can actually see the events around the time where
>things go stale.
>
>I assume that the whole issue is visible throughout the slow progress of
>init towards a working system, so for a start it would be sufficient to add
>something like this into the startup sequence at some point:
>
> mount -t debugfs debugfs /sys/kernel/debug
> echo 0 >/sys/kernel/debug/tracing/tracing_on
>
>The only interesting challange is to get the trace data out of the
>system. The trace is accessible via:
>
> cat /sys/kernel/tracing/trace
>

Thanks for the pointers on tracing. I haven't used it before so that
was very helpful.

>So if your ssh works at some point, that might be an option or you just try
>to store it over NFS (which will be slow, but better than nothing). Maybe
>you have a better idea :)

I've tried a whole bunch of different ways to reproduce the problem
and get the logs out, so far they've all been unsuccessful
(reproducing is easy, getting data out is not).

I have a few more ideas to try, but it's pretty slow work - it's
taking at least 30 minutes per attempt. I'll let you know if I manage
something.

Thanks!
-Brian

>
>Thanks,
>
>	tglx
>

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Regression: Failed boots bisected to 4cd13c21b207 "softirq: Let  ksoftirqd do its job" Brian Starkey <brian.starkey@arm.com> - 2016-11-16 15:00 +0100
  Re: Regression: Failed boots bisected to 4cd13c21b207 "softirq: Let  ksoftirqd do its job" Eric Dumazet <edumazet@google.com> - 2016-11-16 17:00 +0100
    Re: Regression: Failed boots bisected to 4cd13c21b207 "softirq: Let  ksoftirqd do its job" Brian Starkey <brian.starkey@arm.com> - 2016-11-16 19:10 +0100
      Re: Regression: Failed boots bisected to 4cd13c21b207 "softirq: Let  ksoftirqd do its job" Eric Dumazet <edumazet@google.com> - 2016-11-16 19:50 +0100
        Re: Regression: Failed boots bisected to 4cd13c21b207 "softirq: Let  ksoftirqd do its job" Brian Starkey <brian.starkey@arm.com> - 2016-11-16 22:10 +0100
          Re: Regression: Failed boots bisected to 4cd13c21b207 "softirq: Let  ksoftirqd do its job" Brian Starkey <brian.starkey@arm.com> - 2016-11-17 18:10 +0100
            Re: Regression: Failed boots bisected to 4cd13c21b207 "softirq: Let  ksoftirqd do its job" Thomas Gleixner <tglx@linutronix.de> - 2016-11-18 01:50 +0100
              Re: Regression: Failed boots bisected to 4cd13c21b207 "softirq: Let  ksoftirqd do its job" Brian Starkey <brian.starkey@arm.com> - 2016-11-18 21:30 +0100
              Re: Regression: Failed boots bisected to 4cd13c21b207 "softirq: Let  ksoftirqd do its job" Brian Starkey <brian.starkey@arm.com> - 2016-11-22 11:40 +0100
                Re: Regression: Failed boots bisected to 4cd13c21b207 "softirq: Let  ksoftirqd do its job" Eric Dumazet <edumazet@google.com> - 2016-11-22 15:30 +0100
                Re: Regression: Failed boots bisected to 4cd13c21b207 "softirq: Let  ksoftirqd do its job" Brian Starkey <brian.starkey@arm.com> - 2016-11-22 16:30 +0100
                Re: Regression: Failed boots bisected to 4cd13c21b207 "softirq: Let  ksoftirqd do its job" Eric Dumazet <edumazet@google.com> - 2016-11-22 17:20 +0100
                Re: Regression: Failed boots bisected to 4cd13c21b207 "softirq: Let  ksoftirqd do its job" Brian Starkey <brian.starkey@arm.com> - 2016-11-23 19:30 +0100
                Re: Regression: Failed boots bisected to 4cd13c21b207 "softirq: Let  ksoftirqd do its job" Eric Dumazet <edumazet@google.com> - 2016-11-23 21:10 +0100
          Re: Regression: Failed boots bisected to 4cd13c21b207 "softirq: Let  ksoftirqd do its job" Eric Dumazet <edumazet@google.com> - 2016-11-17 18:10 +0100

csiph-web