Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #196474 > unrolled thread

Slow clock on Xen 4.8 after upgrade

Back to article view | Back to linux.debian.user

Contents

  Slow clock on Xen 4.8 after upgrade Mike <debian@norgie.net> - 2018-06-07 21:40 +0200
    Re: Slow clock on Xen 4.8 after upgrade Abdullah Ramazanoğlu <ar018@yahoo.com> - 2018-06-07 22:10 +0200
      Re: Slow clock on Xen 4.8 after upgrade Mike <debian@norgie.net> - 2018-06-11 11:10 +0200

#196474 — Slow clock on Xen 4.8 after upgrade

[Multipart message — attachments visible in raw view] — view raw

Folks,

I've been running a Debian Xen server for a while now.  It was upgraded
to Stretch upon its release and has (I think) ran like that since.  It's
been regularly updated but never rebooted until this weekend which saw
it go from a 4.9.0-3 kernel to 4.9.0-6 and what looks from the apt
history like a few upgrades to the Hypervisor.

Long story short, having ran without issue for around 330 days,
following the reboot, I noticed that the time on the Dom0 and DomUs was
drifting.  After investigating, I noticed that the NTP daemon was
rejecting all of its upstream servers as they were failing sanity.
After a restart and resync of the NTP daemon, the same thing was
happening, after about five minutes, all of the servers would fail
sanity and get rejected.

Upon further investigation, I noticed that the system time was drifting
away from the hardware clock at a fairly alarming rate.  It was
something like 15 minutes a day.

I did take a look at
/sys/devices/system/clocksource/clocksource0/current_clocksource which
was listing tsc.  I understand that can be a little flaky with vCPUS
(although had worked for the past 330 days), so I changed it to xen.
This made no difference at all.  Further investigations led me to: xl
dmesg | grep time which revealed: (XEN) Platform timer is 14.318MHz HPET
but again, something which has worked perfectly well for 330 days.

In the end I used adjtimex -t 10065 -f 3058784 to speed the clock up a
bit and that has allowed NTP to be able to keep it under control.

So the issue with the slow clock appears to have been compenstated for
but I'd be intersted to know what actually caused it.  I've drawn a bit
of a blank on that one and would be grateful if anyone could offer any
thoughts?

Mike.

[toc] | [next] | [standalone]

#196475

On Thu, 7 Jun 2018 19:32:35 +0100 Mike said:

[--8<--]

> In the end I used adjtimex -t 10065 -f 3058784 to speed the clock up a
> bit and that has allowed NTP to be able to keep it under control.
> 
> So the issue with the slow clock appears to have been compenstated for
> but I'd be intersted to know what actually caused it.  I've drawn a bit
> of a blank on that one and would be grateful if anyone could offer any
> thoughts?

Could there be a race condition between hwclock(8) and adjtimex?

IOW, there is a possibility that perhaps there was a drift rate set by hwclock
and this was the cause of excessive drift. Now that adjtimex counter balances
hwclock by an opposite drift rate, it can result in hwclock and adjtimex
fighting against each other. Just a possibility.

I would have checked /etc/adjtime to ensure that no drift ratio set by hwclock.
More info about hwclock drift setting and /etc/adjtime format in hwclock man
page.

Regards
-- 
Abdullah Ramazanoğlu

[toc] | [prev] | [next] | [standalone]

#196553

[Multipart message — attachments visible in raw view] — view raw

On Thu, Jun 07, 2018 at 11:06:12PM +0300, Abdullah Ramazanoğlu wrote:
> On Thu, 7 Jun 2018 19:32:35 +0100 Mike said:
> 
> [--8<--]
> 
> > In the end I used adjtimex -t 10065 -f 3058784 to speed the clock up a
> > bit and that has allowed NTP to be able to keep it under control.
> > 
> > So the issue with the slow clock appears to have been compenstated for
> > but I'd be intersted to know what actually caused it.  I've drawn a bit
> > of a blank on that one and would be grateful if anyone could offer any
> > thoughts?
> 
> Could there be a race condition between hwclock(8) and adjtimex?
> 
> IOW, there is a possibility that perhaps there was a drift rate set by hwclock
> and this was the cause of excessive drift. Now that adjtimex counter balances
> hwclock by an opposite drift rate, it can result in hwclock and adjtimex
> fighting against each other. Just a possibility.
> 
> I would have checked /etc/adjtime to ensure that no drift ratio set by hwclock.
> More info about hwclock drift setting and /etc/adjtime format in hwclock man
> page.
>

Thanks for the reply.

I did do some fiddling in /etc/adjtime (setting the drift ratio to 0 to
see if that resolved the issue, although it was unclear to me from the
docs I read how and when this file gets read (I believe it's hwclock
that reads it but when, other than on boot?)  I did keep  a copy of the
original before I fiddled with it:

3.619663 1452036888 0.000000
1452036888
UTC

I believe this means that it hasn't been updated since the begining of
time and that there's a drift of 3s a day between the hardware and
system clocks?  I assume this should not account for the ~15 mins a day
drift I was seeing?

Regards,
Mike. 

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web