Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1209127

Re: [PATCH 8/9] clocksource: Improve unstable clocksource detection

Show all headers | View raw

On Mon, 17 Aug 2015, John Stultz wrote:
> On Mon, Aug 17, 2015 at 3:04 PM, Thomas Gleixner <tglx@linutronix.de> wrote:
> > On Mon, 17 Aug 2015, John Stultz wrote:
> >
> >> From: Shaohua Li <shli@fb.com>
> >>
> >> >From time to time we saw TSC is marked as unstable in our systems, while
> >
> > Stray '>'
> >
> >> the CPUs declare to have stable TSC. Looking at the clocksource unstable
> >> detection, there are two problems:
> >> - watchdog clock source wrap. HPET is the most common watchdog clock
> >>   source. It's 32-bit and runs in 14.3Mhz. That means the hpet counter
> >>   can wrap in about 5 minutes.
> >> - threshold isn't scaled against interval. The threshold is 0.0625s in
> >>   0.5s interval. What if the actual interval is bigger than 0.5s?
> >>
> >> The watchdog runs in a timer bh, so hard/soft irq can defer its running.
> >> Heavy network stack softirq can hog a cpu. IPMI driver can disable
> >> interrupt for a very long time.
> >
> > And they hold off the timer softirq for more than a second? Don't you
> > think that's the problem which needs to be fixed?
> 
> Though this is an issue I've experienced (and tried unsuccessfully to
> fix in a more complicated way) with the RT kernel, where high priority
> tasks blocked the watchdog long enough that we'd disqualify the TSC.

Did it disqualify the watchdog due to HPET wraparounds (5 minutes) or
due to the fixed threshold being applied?

> > So 'fixing' the watchdog is the wrong approach. Fixing the stuff which
> > prevents the watchdog to run is the proper thing to do.
> 
> I'm not sure here. I feel like these delay-caused false positives
> (I've seen similar reports w/ VMs being stalled) are more common then
> one-off SMI TSC skews.

Yes, they are more common, but the other issues are reality as well.

> There are hard lines in the timekeeping code, where we do say: Don't
> delay us past X or we can't really handle it, but in this case, the
> main clocksource is fine and the limit is being caused by the
> watchdog. So I think some sort of a solution to remove this
> restriction would be good. We don't want to needlessly punish fine
> hardware because our checks for bad hardware add extra restrictions.

No argument here. Though fine hardware has an escape route already to
avoid the watchdog business alltogether (tsc=reliable on the command
line).

> That said, I agree the "should"s and other vague qualifiers in the
> commit description you point out should have more specifics to back
> things up. And I'm fine delaying this (and the follow-on) patch until
> those details are provided.

Fair enough.

Thanks,

	tglx


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH 0/9] Time items for 4.3 John Stultz <john.stultz@linaro.org> - 2015-08-17 22:50 +0200
  [PATCH 2/9] time: Fix nanosecond file time rounding in timespec_trunc() John Stultz <john.stultz@linaro.org> - 2015-08-17 22:50 +0200
    Re: [PATCH 2/9] time: Fix nanosecond file time rounding in  timespec_trunc() Thomas Gleixner <tglx@linutronix.de> - 2015-08-18 00:20 +0200
  [PATCH 6/9] time: Introduce current_kernel_time64() John Stultz <john.stultz@linaro.org> - 2015-08-17 22:50 +0200
  [PATCH 1/9] timer_list: Add the base offset so remaining nsecs are accurate for non monotonic timers John Stultz <john.stultz@linaro.org> - 2015-08-17 22:50 +0200
    Re: [PATCH 1/9] timer_list: Add the base offset so remaining nsecs  are accurate for non monotonic timers Shuah Khan <shuahkh@osg.samsung.com> - 2015-08-17 23:10 +0200
      Re: [PATCH 1/9] timer_list: Add the base offset so remaining nsecs  are accurate for non monotonic timers John Stultz <john.stultz@linaro.org> - 2015-08-17 23:10 +0200
      Re: [PATCH 1/9] timer_list: Add the base offset so remaining nsecs  are accurate for non monotonic timers Shuah Khan <shuahkh@osg.samsung.com> - 2015-08-17 23:10 +0200
  [PATCH 3/9] time: Always make sure wall_to_monotonic isn't positive John Stultz <john.stultz@linaro.org> - 2015-08-17 22:50 +0200
  [PATCH 7/9] time: Introduce timespec64_to_jiffies()/jiffies_to_timespec64() John Stultz <john.stultz@linaro.org> - 2015-08-17 22:50 +0200
  [PATCH 4/9] time: Add the common weak version of update_persistent_clock() John Stultz <john.stultz@linaro.org> - 2015-08-17 22:50 +0200
  [PATCH 8/9] clocksource: Improve unstable clocksource detection John Stultz <john.stultz@linaro.org> - 2015-08-17 22:50 +0200
    Re: [PATCH 8/9] clocksource: Improve unstable clocksource  detection Thomas Gleixner <tglx@linutronix.de> - 2015-08-18 00:10 +0200
      Re: [PATCH 8/9] clocksource: Improve unstable clocksource detection John Stultz <john.stultz@linaro.org> - 2015-08-18 00:20 +0200
        Re: [PATCH 8/9] clocksource: Improve unstable clocksource detection Shaohua Li <shli@fb.com> - 2015-08-18 05:00 +0200
          Re: [PATCH 8/9] clocksource: Improve unstable clocksource detection John Stultz <john.stultz@linaro.org> - 2015-08-18 05:40 +0200
          Re: [PATCH 8/9] clocksource: Improve unstable clocksource  detection Thomas Gleixner <tglx@linutronix.de> - 2015-08-18 11:00 +0200
        Re: [PATCH 8/9] clocksource: Improve unstable clocksource  detection Thomas Gleixner <tglx@linutronix.de> - 2015-08-18 10:40 +0200
          Re: [PATCH 8/9] clocksource: Improve unstable clocksource detection John Stultz <john.stultz@linaro.org> - 2015-08-18 19:50 +0200
            Re: [PATCH 8/9] clocksource: Improve unstable clocksource  detection Thomas Gleixner <tglx@linutronix.de> - 2015-08-18 21:30 +0200
              Re: [PATCH 8/9] clocksource: Improve unstable clocksource  detection Thomas Gleixner <tglx@linutronix.de> - 2015-08-18 22:20 +0200
              Re: [PATCH 8/9] clocksource: Improve unstable clocksource detection John Stultz <john.stultz@linaro.org> - 2015-08-18 22:20 +0200

csiph-web