Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1158144 > unrolled thread

[BUG, bisect] hrtimer: severe lag after suspend & resume

Back to article view | Back to linux.kernel

Contents

  [BUG, bisect] hrtimer: severe lag after suspend & resume Jeremiah Mahler <jmmahler@gmail.com> - 2015-06-04 03:00 +0200
    Re: [BUG, bisect] hrtimer: severe lag after suspend & resume Thomas Gleixner <tglx@linutronix.de> - 2015-06-04 13:30 +0200
    Re: [BUG, bisect] hrtimer: severe lag after suspend & resume Jeremiah Mahler <jmmahler@gmail.com> - 2015-06-05 02:10 +0200

#1158144 — [BUG, bisect] hrtimer: severe lag after suspend & resume

all,

After a fresh boot, the Chrome web browser behaves normally.  Pages
load quickly and scroll fast.  Even image heavy sites such as
images.google.com work fine.  However, after a suspend and resume
cycle, Chrome becomes very slow.  Pages take ten seconds or more to
load.  The scroll bars and buttons are almost completely
unresponsive.  Interestingly, I can run Firefox on the same sites
and it has no issue whatsoever.

I have bisected the kernel and found that the following commit
introduced the bug.  It is present in the latest linux-next (20150602).

  From 868a3e915f7f5eba8f8cb4f7da2276760807c51c Mon Sep 17 00:00:00 2001
  From: Thomas Gleixner <tglx@linutronix.de>
  Date: Tue, 14 Apr 2015 21:08:37 +0000
  Subject: [PATCH] hrtimer: Make offset update smarter
  
  On every tick/hrtimer interrupt we update the offset variables of the
  clock bases. That's silly because these offsets change very seldom.
  
  Add a sequence counter to the time keeping code which keeps track of
  the offset updates (clock_was_set()). Have a sequence cache in the
  hrtimer cpu bases to evaluate whether the offsets must be updated or
  not. This allows us later to avoid pointless cacheline pollution.
  
  Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
  Reviewed-by: Preeti U Murthy <preeti@linux.vnet.ibm.com>
  Acked-by: Peter Zijlstra <peterz@infradead.org>
  Cc: Viresh Kumar <viresh.kumar@linaro.org>
  Cc: Marcelo Tosatti <mtosatti@redhat.com>
  Cc: Frederic Weisbecker <fweisbec@gmail.com>
  Cc: John Stultz <john.stultz@linaro.org>
  Link: http://lkml.kernel.org/r/20150414203501.132820245@linutronix.de
  Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
  Cc: John Stultz <john.stultz@linaro.org>
  ---
   include/linux/hrtimer.h             |  4 ++--
   include/linux/timekeeper_internal.h |  2 ++
   kernel/time/hrtimer.c               |  3 ++-
   kernel/time/timekeeping.c           | 23 ++++++++++++++++-------
   kernel/time/timekeeping.h           |  7 ++++---
   5 files changed, 26 insertions(+), 13 deletions(-)

-- 
- Jeremiah Mahler
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1158478

On Wed, 3 Jun 2015, Jeremiah Mahler wrote:
> After a fresh boot, the Chrome web browser behaves normally.  Pages
> load quickly and scroll fast.  Even image heavy sites such as
> images.google.com work fine.  However, after a suspend and resume
> cycle, Chrome becomes very slow.  Pages take ten seconds or more to
> load.  The scroll bars and buttons are almost completely
> unresponsive.  Interestingly, I can run Firefox on the same sites
> and it has no issue whatsoever.

Weird.
 
> I have bisected the kernel and found that the following commit
> introduced the bug.  It is present in the latest linux-next (20150602).
> 
>   From 868a3e915f7f5eba8f8cb4f7da2276760807c51c Mon Sep 17 00:00:00 2001
>   From: Thomas Gleixner <tglx@linutronix.de>
>   Date: Tue, 14 Apr 2015 21:08:37 +0000
>   Subject: [PATCH] hrtimer: Make offset update smarter
>   
>   On every tick/hrtimer interrupt we update the offset variables of the
>   clock bases. That's silly because these offsets change very seldom.
>   
>   Add a sequence counter to the time keeping code which keeps track of
>   the offset updates (clock_was_set()). Have a sequence cache in the
>   hrtimer cpu bases to evaluate whether the offsets must be updated or
>   not. This allows us later to avoid pointless cacheline pollution.

I had to wrap my head around that for quite a while, but I think I
have decoded the issue. Can you please test the patch below whether it
solves your problem?

Thanks,

	tglx

------------------------>

diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index 278d4b36fd94..e9dfcd0b8c41 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -1492,6 +1492,12 @@ static void init_hrtimers_cpu(int cpu)
 
 	cpu_base->cpu = cpu;
 	hrtimer_init_hres(cpu_base);
+	/*
+	 * Force an update by setting the clock was set sequence to an
+	 * odd value.
+	 */
+	cpu_base->clock_was_set_seq = 1;
+	hrtimer_update_base(cpu_base);
 }
 
 #ifdef CONFIG_HOTPLUG_CPU
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index 90ed5db67c1d..c97710137a9e 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -593,7 +593,7 @@ static void timekeeping_update(struct timekeeper *tk, unsigned int action)
 	update_fast_timekeeper(&tk->tkr_raw,  &tk_fast_raw);
 
 	if (action & TK_CLOCK_WAS_SET)
-		tk->clock_was_set_seq++;
+		tk->clock_was_set_seq += 2;
 }
 
 /**
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1158895

John,

On Thu, Jun 04, 2015 at 03:54:35PM -0700, John Stultz wrote:
> On Wed, Jun 3, 2015 at 5:56 PM, Jeremiah Mahler <jmmahler@gmail.com> wrote:
[...]
> 
> 
> So I suspect the problem is the change to clock_was_set_seq in
> timekeeping_update is done prior to mirroring the time state to the
> shadow-timekeeper. Thus the next time we do update_wall_time() the
> updated sequence is overwritten by whats in the shadow copy. The
> attached patch moving the modification up seems to avoid the issue for
> me.
> 
> Thomas:  Looking at the problematic change, I'm not a big fan of it.
> Caching timekeeping state here in the hrtimer code has been a source
> of bugs in the past, and I'm not sure I see how avoiding copying
> 24bytes is that big of a win. Especially since it adds more state to
> the timekeeper and hrtimer base that we have to read and mange.
> Personally I'd prefer a revert to my fix.
> 
> thanks
> -john

> diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
> index 90ed5db..53be796 100644
> --- a/kernel/time/timekeeping.c
> +++ b/kernel/time/timekeeping.c
> @@ -580,6 +580,9 @@ static void timekeeping_update(struct timekeeper *tk, unsigned int action)
>  		ntp_clear();
>  	}
>  
> +	if (action & TK_CLOCK_WAS_SET)
> +		tk->clock_was_set_seq++;
> +
>  	tk_update_ktime_data(tk);
>  
>  	update_vsyscall(tk);
> @@ -591,9 +594,6 @@ static void timekeeping_update(struct timekeeper *tk, unsigned int action)
>  
>  	update_fast_timekeeper(&tk->tkr_mono, &tk_fast_mono);
>  	update_fast_timekeeper(&tk->tkr_raw,  &tk_fast_raw);
> -
> -	if (action & TK_CLOCK_WAS_SET)
> -		tk->clock_was_set_seq++;
>  }
>  
>  /**

That patch fixes the problem for me.

Thanks John.

-- 
- Jeremiah Mahler
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web