Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1485326 > unrolled thread

[RFC v7 00/23] adapt clockevents frequencies to mono clock

Started byNicolai Stange <nicstange@gmail.com>
First post2016-09-16 22:20 +0200
Last post2016-09-23 00:40 +0200
Articles 7 on this page of 27 — 2 participants

Back to article view | Back to linux.kernel

Contents

  [RFC v7 00/23] adapt clockevents frequencies to mono clock Nicolai Stange <nicstange@gmail.com> - 2016-09-16 22:20 +0200
    [RFC v7 14/23] clockevents: decouple ->max_delta_ns from ->max_delta_ticks Nicolai Stange <nicstange@gmail.com> - 2016-09-16 22:20 +0200
    [RFC v7 12/23] many clockevent drivers: don't set ->min_delta_ns and ->max_delta_ns Nicolai Stange <nicstange@gmail.com> - 2016-09-16 22:20 +0200
    [RFC v7 09/23] arch/x86/platform/uv/uv_time: set ->min_delta_ticks and ->max_delta_ticks Nicolai Stange <nicstange@gmail.com> - 2016-09-16 22:20 +0200
    [RFC v7 20/23] clockevents: purge ->min_delta_ns Nicolai Stange <nicstange@gmail.com> - 2016-09-16 22:20 +0200
    [RFC v7 21/23] clockevents: initial support for mono to raw time conversion Nicolai Stange <nicstange@gmail.com> - 2016-09-16 22:20 +0200
    [RFC v7 10/23] arch/tile/kernel/time: set ->min_delta_ticks and ->max_delta_ticks Nicolai Stange <nicstange@gmail.com> - 2016-09-16 22:20 +0200
    [RFC v7 06/23] clockevents: make clockevents_config() static Nicolai Stange <nicstange@gmail.com> - 2016-09-16 22:20 +0200
    [RFC v7 03/23] clocksource: em_sti: split clock prepare and enable steps Nicolai Stange <nicstange@gmail.com> - 2016-09-16 22:20 +0200
    [RFC v7 15/23] clockevents: do comparison of delta against minimum in terms of cycles Nicolai Stange <nicstange@gmail.com> - 2016-09-16 22:20 +0200
    [RFC v7 13/23] clockevents: introduce CLOCK_EVT_FEAT_NO_ADJUST flag Nicolai Stange <nicstange@gmail.com> - 2016-09-16 22:20 +0200
    [RFC v7 22/23] clockevents: make setting of ->mult and ->mult_adjusted atomic Nicolai Stange <nicstange@gmail.com> - 2016-09-16 22:20 +0200
    [RFC v7 04/23] clocksource: em_sti: compute rate before registration Nicolai Stange <nicstange@gmail.com> - 2016-09-16 22:20 +0200
    [RFC v7 16/23] clockevents: clockevents_program_min_delta(): don't set ->next_event Nicolai Stange <nicstange@gmail.com> - 2016-09-16 22:20 +0200
    [RFC v7 07/23] many clockevent drivers: set ->min_delta_ticks and ->max_delta_ticks Nicolai Stange <nicstange@gmail.com> - 2016-09-16 22:20 +0200
    [RFC v7 19/23] timer_list: print_tickdevice(): calculate ->min_delta_ns dynamically Nicolai Stange <nicstange@gmail.com> - 2016-09-16 22:20 +0200
    [RFC v7 11/23] clockevents: always initialize ->min_delta_ns and ->max_delta_ns Nicolai Stange <nicstange@gmail.com> - 2016-09-16 22:20 +0200
    [RFC v7 18/23] clockevents: min delta increment: calculate min_delta_ns from ticks Nicolai Stange <nicstange@gmail.com> - 2016-09-16 22:20 +0200
    [RFC v7 05/23] clocksource: h8300_timer8: don't reset rate in ->set_state_oneshot() Nicolai Stange <nicstange@gmail.com> - 2016-09-16 22:20 +0200
    [RFC v7 17/23] clockevents: use ->min_delta_ticks_adjusted to program minimum delta Nicolai Stange <nicstange@gmail.com> - 2016-09-16 22:20 +0200
    [RFC v7 23/23] timekeeping: inform clockevents about freq adjustments Nicolai Stange <nicstange@gmail.com> - 2016-09-16 22:30 +0200
    Re: [RFC v7 00/23] adapt clockevents frequencies to mono clock Thomas Gleixner <tglx@linutronix.de> - 2016-09-20 23:00 +0200
      Re: [RFC v7 00/23] adapt clockevents frequencies to mono clock Nicolai Stange <nicstange@gmail.com> - 2016-09-21 01:10 +0200
        Re: [RFC v7 00/23] adapt clockevents frequencies to mono clock Thomas Gleixner <tglx@linutronix.de> - 2016-09-21 01:40 +0200
          Re: [RFC v7 00/23] adapt clockevents frequencies to mono clock Nicolai Stange <nicstange@gmail.com> - 2016-09-21 16:10 +0200
            Re: [RFC v7 00/23] adapt clockevents frequencies to mono clock Thomas Gleixner <tglx@linutronix.de> - 2016-09-22 23:50 +0200
              Re: [RFC v7 00/23] adapt clockevents frequencies to mono clock Nicolai Stange <nicstange@gmail.com> - 2016-09-23 00:40 +0200

Page 2 of 2 — ← Prev page 1 [2]

#1485351 — [RFC v7 23/23] timekeeping: inform clockevents about freq adjustments

FromNicolai Stange <nicstange@gmail.com>
Date2016-09-16 22:30 +0200
Subject[RFC v7 23/23] timekeeping: inform clockevents about freq adjustments
Message-ID<si7VL-1eB-5@gated-at.bofh.it>
In reply to#1485326
Upon adjustments of the monotonic clock's frequencies from the
timekeeping core, the clockevents devices' ->mult_adjusted should be
changed accordingly, too.

Introduce clockevents_adjust_all_freqs() which traverses all registered
clockevent devices and, if the CLOCK_EVT_FEAT_NO_ADJUST flag is not set,
recalculates their ->mult_adjusted based on the monotonic clock's current
frequency.

Call clockevents_adjust_all_freqs() from timekeeping_freqadjust().

Note that it might look like as if timekeeping_apply_adjustment() was the
more natural candidate to trigger the clockevent devices' frequency updates
from: it's the single place where the mono clock's ->mult is changed.
However, timekeeping_apply_adjustment() is also invoked for the
on-off-controlled adjustments made to the mono clock's ->mult from
timekeeping_adjust(). These adjustments are very small in magnitude and,
more importantly, exhibit some oscillatory behaviour once the NTP error
becomes small. We don't want the clockevent devices' ->mult values to
follow these oscillations because they're negligible and because the
process of updating them would periodically destroy what
clockevents_increase_min_delta() might have built up.

Signed-off-by: Nicolai Stange <nicstange@gmail.com>
---
 kernel/time/clockevents.c   | 33 +++++++++++++++++++++++++++++++++
 kernel/time/tick-internal.h |  5 +++++
 kernel/time/timekeeping.c   |  2 ++
 3 files changed, 40 insertions(+)

diff --git a/kernel/time/clockevents.c b/kernel/time/clockevents.c
index b121932..e009add 100644
--- a/kernel/time/clockevents.c
+++ b/kernel/time/clockevents.c
@@ -626,6 +626,39 @@ void __clockevents_adjust_freq(struct clock_event_device *dev)
 							mult_cs_raw);
 }
 
+void clockevents_adjust_all_freqs(u32 mult_cs_mono, u32 mult_cs_raw)
+{
+	u32 last_mult_raw = 0, last_shift = 0, last_mult_adjusted = 0;
+	u32 mult_raw, shift;
+	unsigned long flags;
+	struct clock_event_device *dev;
+
+	raw_spin_lock_irqsave(&clockevents_lock, flags);
+	list_for_each_entry(dev, &clockevent_devices, list) {
+		if (!(dev->features & CLOCK_EVT_FEAT_ONESHOT) ||
+		     (dev->features & CLOCK_EVT_FEAT_NO_ADJUST))
+			continue;
+
+		/*
+		 * The cached last_mult_adjusted is only valid if
+		 * shift == last_shift. Otherwise, it could exceed
+		 * what is allowed by ->max_delta_ns.
+		 */
+		mult_raw = dev->mult;
+		shift = dev->shift;
+		if (mult_raw != last_mult_raw || shift != last_shift) {
+			last_mult_raw = mult_raw;
+			last_shift = shift;
+			last_mult_adjusted =
+				__clockevents_calc_adjust_freq(mult_raw,
+							mult_cs_mono,
+							mult_cs_raw);
+		}
+		dev->mult_adjusted = last_mult_adjusted;
+	}
+	raw_spin_unlock_irqrestore(&clockevents_lock, flags);
+}
+
 int __clockevents_update_freq(struct clock_event_device *dev, u32 freq)
 {
 	clockevents_config(dev, freq);
diff --git a/kernel/time/tick-internal.h b/kernel/time/tick-internal.h
index 0b29d23..2d97c42 100644
--- a/kernel/time/tick-internal.h
+++ b/kernel/time/tick-internal.h
@@ -56,6 +56,7 @@ extern int clockevents_program_event(struct clock_event_device *dev,
 				     ktime_t expires, bool force);
 extern void clockevents_handle_noop(struct clock_event_device *dev);
 extern int __clockevents_update_freq(struct clock_event_device *dev, u32 freq);
+extern void clockevents_adjust_all_freqs(u32 mult_cs_mono, u32 mult_cs_raw);
 extern void timekeeping_get_mono_mult(u32 *mult_cs_mono, u32 *mult_cs_raw);
 extern ssize_t sysfs_get_uname(const char *buf, char *dst, size_t cnt);
 
@@ -95,6 +96,10 @@ static inline void tick_set_periodic_handler(struct clock_event_device *dev, int
 #else /* !GENERIC_CLOCKEVENTS: */
 static inline void tick_suspend(void) { }
 static inline void tick_resume(void) { }
+
+static inline void clockevents_adjust_all_freqs(u32 mult_cs_mono,
+						u32 mult_cs_raw)
+{}
 #endif /* !GENERIC_CLOCKEVENTS */
 
 /* Oneshot related functions */
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index 7ddca9e..f55926b 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -1910,6 +1910,8 @@ static __always_inline void timekeeping_freqadjust(struct timekeeper *tk,
 
 	/* scale the corrections */
 	timekeeping_apply_adjustment(tk, offset, negative, adj_scale);
+	clockevents_adjust_all_freqs(tk->tkr_mono.mult,
+				tk->tkr_mono.clock->mult);
 }
 
 /*
-- 
2.10.0

[toc] | [prev] | [next] | [standalone]

#1487632

FromThomas Gleixner <tglx@linutronix.de>
Date2016-09-20 23:00 +0200
Message-ID<sjAj0-81b-27@gated-at.bofh.it>
In reply to#1485326
On Fri, 16 Sep 2016, Nicolai Stange wrote:

> Goal: avoid programming ced devices too early for large deltas, for
>       details, c.f. the description of [21/23].
> 
> [21-23/23] Actually do the frequency adjustments.
> 
> Tested on x86_64 and next-20160916.

Have you ever measured the overhead of the extra work which has to be done
in clockevents_adjust_all_freqs() ?

Thanks,

	tglx

[toc] | [prev] | [next] | [standalone]

#1487713

FromNicolai Stange <nicstange@gmail.com>
Date2016-09-21 01:10 +0200
Message-ID<sjCkN-13s-15@gated-at.bofh.it>
In reply to#1487632
Thomas Gleixner <tglx@linutronix.de> writes:

> On Fri, 16 Sep 2016, Nicolai Stange wrote:
>
>> Goal: avoid programming ced devices too early for large deltas, for
>>       details, c.f. the description of [21/23].
>> 
>> [21-23/23] Actually do the frequency adjustments.
>> 
>> Tested on x86_64 and next-20160916.
>
> Have you ever measured the overhead of the extra work which has to be done
> in clockevents_adjust_all_freqs() ?

Not exactly, I had a look at its invocation frequency which seems to
decay exponentially with uptime, presumably because the NTP error
approaches zero.

However, I've just gathered a function_graph ftrace on my Intel
i7-4800MQ (Haswell, 8HTs):

# tracer: function_graph
#
#     TIME        CPU  DURATION                  FUNCTION CALLS
#      |          |     |   |                     |   |   |   |
   85.287027 |   0)   0.899 us    |  clockevents_adjust_all_freqs();
   85.288026 |   0)   0.759 us    |  clockevents_adjust_all_freqs();
   85.289026 |   0)   0.735 us    |  clockevents_adjust_all_freqs();
   85.290026 |   0)   0.671 us    |  clockevents_adjust_all_freqs();
  149.503656 |   2)   2.477 us    |  clockevents_adjust_all_freqs();
  149.507660 |   2)   2.308 us    |  clockevents_adjust_all_freqs();
  149.511658 |   2)   2.651 us    |  clockevents_adjust_all_freqs();
  149.545660 |   0)   2.268 us    |  clockevents_adjust_all_freqs();
  149.564211 |   2)   2.321 us    |  clockevents_adjust_all_freqs();
  214.351899 |   2)   1.520 us    |  clockevents_adjust_all_freqs();
  214.354935 |   0)   1.053 us    |  clockevents_adjust_all_freqs();
  279.026205 |   0)   2.289 us    |  clockevents_adjust_all_freqs();
  279.030195 |   0)   2.190 us    |  clockevents_adjust_all_freqs();
  279.034196 |   0)   2.381 us    |  clockevents_adjust_all_freqs();
  279.047492 |   2)   2.390 us    |  clockevents_adjust_all_freqs();
  344.250356 |   1)   2.727 us    |  clockevents_adjust_all_freqs();
  408.879538 |   1)   2.235 us    |  clockevents_adjust_all_freqs();
  473.125730 |   6)   1.513 us    |  clockevents_adjust_all_freqs();
  473.129731 |   6)   1.650 us    |  clockevents_adjust_all_freqs();
  538.387891 |   3)   2.305 us    |  clockevents_adjust_all_freqs();
  538.391890 |   3)   2.300 us    |  clockevents_adjust_all_freqs();
  668.257162 |   3)   2.691 us    |  clockevents_adjust_all_freqs();
  668.261162 |   3)   2.306 us    |  clockevents_adjust_all_freqs();
  733.459261 |   0)   1.066 us    |  clockevents_adjust_all_freqs();
  733.463261 |   0)   1.233 us    |  clockevents_adjust_all_freqs();
  733.467263 |   1)   1.382 us    |  clockevents_adjust_all_freqs();
  863.398561 |   2)   2.218 us    |  clockevents_adjust_all_freqs();
  863.402552 |   2)   2.792 us    |  clockevents_adjust_all_freqs();
 1122.210001 |   3)   2.259 us    |  clockevents_adjust_all_freqs();
 1122.214004 |   3)   2.165 us    |  clockevents_adjust_all_freqs();
 1381.283287 |   2)   1.944 us    |  clockevents_adjust_all_freqs();
 1895.664008 |   2)   1.940 us    |  clockevents_adjust_all_freqs();
 1895.668009 |   2)   2.041 us    |  clockevents_adjust_all_freqs();
 2930.385388 |   0)   1.067 us    |  clockevents_adjust_all_freqs();
 2930.386390 |   5)   1.208 us    |  clockevents_adjust_all_freqs();


Thanks,

Nicolai

[toc] | [prev] | [next] | [standalone]

#1487720

FromThomas Gleixner <tglx@linutronix.de>
Date2016-09-21 01:40 +0200
Message-ID<sjCNP-1cK-15@gated-at.bofh.it>
In reply to#1487713
On Wed, 21 Sep 2016, Nicolai Stange wrote:
> Thomas Gleixner <tglx@linutronix.de> writes:
> > Have you ever measured the overhead of the extra work which has to be done
> > in clockevents_adjust_all_freqs() ?
> 
> Not exactly, I had a look at its invocation frequency which seems to
> decay exponentially with uptime, presumably because the NTP error
> approaches zero.
> 
> However, I've just gathered a function_graph ftrace on my Intel
> i7-4800MQ (Haswell, 8HTs):
> 
> #     TIME        CPU  DURATION                  FUNCTION CALLS
> #      |          |     |   |                     |   |   |   |
>    85.287027 |   0)   0.899 us    |  clockevents_adjust_all_freqs();
>    85.288026 |   0)   0.759 us    |  clockevents_adjust_all_freqs();
>    85.289026 |   0)   0.735 us    |  clockevents_adjust_all_freqs();
>    85.290026 |   0)   0.671 us    |  clockevents_adjust_all_freqs();
>   149.503656 |   2)   2.477 us    |  clockevents_adjust_all_freqs();

That's not that bad. Though I'd like to see numbers for ARM (especially the
less powerful SoCs) as well.

Thanks,

	tglx

[toc] | [prev] | [next] | [standalone]

#1488146

FromNicolai Stange <nicstange@gmail.com>
Date2016-09-21 16:10 +0200
Message-ID<sjQnM-1xc-17@gated-at.bofh.it>
In reply to#1487720
Thomas Gleixner <tglx@linutronix.de> writes:

> On Wed, 21 Sep 2016, Nicolai Stange wrote:
>> Thomas Gleixner <tglx@linutronix.de> writes:
>> > Have you ever measured the overhead of the extra work which has to be done
>> > in clockevents_adjust_all_freqs() ?
>> 
>> Not exactly, I had a look at its invocation frequency which seems to
>> decay exponentially with uptime, presumably because the NTP error
>> approaches zero.
>> 
>> However, I've just gathered a function_graph ftrace on my Intel
>> i7-4800MQ (Haswell, 8HTs):
>> 
>> #     TIME        CPU  DURATION                  FUNCTION CALLS
>> #      |          |     |   |                     |   |   |   |
>>    85.287027 |   0)   0.899 us    |  clockevents_adjust_all_freqs();
>>    85.288026 |   0)   0.759 us    |  clockevents_adjust_all_freqs();
>>    85.289026 |   0)   0.735 us    |  clockevents_adjust_all_freqs();
>>    85.290026 |   0)   0.671 us    |  clockevents_adjust_all_freqs();
>>   149.503656 |   2)   2.477 us    |  clockevents_adjust_all_freqs();
>
> That's not that bad. Though I'd like to see numbers for ARM (especially the
> less powerful SoCs) as well.

On a Raspberry Pi 2B (bcm2836, ARMv7) with CONFIG_SMP=y, the mean over
~5300 samples is 5.14+/-1.04us with a max of 11.15us.

Unfortunately, the invocation frequency doesn't calm down as much as it
did on x86_64: after an uptime of 45min, I'm still seeing approximately
one invocation per second. Right after boot, it was ~3/s.

Thanks,

Nicolai

[toc] | [prev] | [next] | [standalone]

#1489581

FromThomas Gleixner <tglx@linutronix.de>
Date2016-09-22 23:50 +0200
Message-ID<skk2u-3f3-17@gated-at.bofh.it>
In reply to#1488146
On Wed, 21 Sep 2016, Nicolai Stange wrote:
> Thomas Gleixner <tglx@linutronix.de> writes:
> 
> > On Wed, 21 Sep 2016, Nicolai Stange wrote:
> >> Thomas Gleixner <tglx@linutronix.de> writes:
> >> > Have you ever measured the overhead of the extra work which has to be done
> >> > in clockevents_adjust_all_freqs() ?
> >> 
> >> Not exactly, I had a look at its invocation frequency which seems to
> >> decay exponentially with uptime, presumably because the NTP error
> >> approaches zero.
> >> 
> >> However, I've just gathered a function_graph ftrace on my Intel
> >> i7-4800MQ (Haswell, 8HTs):
> >> 
> >> #     TIME        CPU  DURATION                  FUNCTION CALLS
> >> #      |          |     |   |                     |   |   |   |
> >>    85.287027 |   0)   0.899 us    |  clockevents_adjust_all_freqs();
> >>    85.288026 |   0)   0.759 us    |  clockevents_adjust_all_freqs();
> >>    85.289026 |   0)   0.735 us    |  clockevents_adjust_all_freqs();
> >>    85.290026 |   0)   0.671 us    |  clockevents_adjust_all_freqs();
> >>   149.503656 |   2)   2.477 us    |  clockevents_adjust_all_freqs();
> >
> > That's not that bad. Though I'd like to see numbers for ARM (especially the
> > less powerful SoCs) as well.
> 
> On a Raspberry Pi 2B (bcm2836, ARMv7) with CONFIG_SMP=y, the mean over
> ~5300 samples is 5.14+/-1.04us with a max of 11.15us.

So why is the variance that high? You have an outlier on that intel as well
which might be caused by NMI, but it might also be a systematic issue
depending on the input parameters. 11 us on that ARM worries me.

Thanks,

	tglx

[toc] | [prev] | [next] | [standalone]

#1489624

FromNicolai Stange <nicstange@gmail.com>
Date2016-09-23 00:40 +0200
Message-ID<skkOS-3Qd-23@gated-at.bofh.it>
In reply to#1489581
Thomas Gleixner <tglx@linutronix.de> writes:

> On Wed, 21 Sep 2016, Nicolai Stange wrote:
>> Thomas Gleixner <tglx@linutronix.de> writes:
>> 
>> > On Wed, 21 Sep 2016, Nicolai Stange wrote:
>> >> Thomas Gleixner <tglx@linutronix.de> writes:
>> >> > Have you ever measured the overhead of the extra work which has to be done
>> >> > in clockevents_adjust_all_freqs() ?
>> >> 
>> >> Not exactly, I had a look at its invocation frequency which seems to
>> >> decay exponentially with uptime, presumably because the NTP error
>> >> approaches zero.
>> >> 
>> >> However, I've just gathered a function_graph ftrace on my Intel
>> >> i7-4800MQ (Haswell, 8HTs):
>> >> 
>> >> #     TIME        CPU  DURATION                  FUNCTION CALLS
>> >> #      |          |     |   |                     |   |   |   |
>> >>    85.287027 |   0)   0.899 us    |  clockevents_adjust_all_freqs();
>> >>    85.288026 |   0)   0.759 us    |  clockevents_adjust_all_freqs();
>> >>    85.289026 |   0)   0.735 us    |  clockevents_adjust_all_freqs();
>> >>    85.290026 |   0)   0.671 us    |  clockevents_adjust_all_freqs();
>> >>   149.503656 |   2)   2.477 us    |  clockevents_adjust_all_freqs();
>> >
>> > That's not that bad. Though I'd like to see numbers for ARM (especially the
>> > less powerful SoCs) as well.
>> 
>> On a Raspberry Pi 2B (bcm2836, ARMv7) with CONFIG_SMP=y, the mean over
>> ~5300 samples is 5.14+/-1.04us with a max of 11.15us.
>
> So why is the variance that high?

I think this is because the histogram has got two peaks, c.f. [1]

Also, the 11us maximum is not isolated but a flat tail is reaching to
this point which I admittedly can't explain.

> You have an outlier on that intel as well which might be caused by
> NMI, but it might also be a systematic issue depending on the input
> parameters.

AFACIT, the "algorithmic" runtime should be constant per CED, so it
should not be dependent on any input parameters.

> 11 us on that ARM worries me.

I'll try to do some more tracing tomorrow in order to get the reason for
that histogram's long tail. But I have to admit that I don't really know
what to look for except for NMIs. Any hints?
What might be remarkable in this context is that the dataset's min is
at 2.24us. Perhaps I'm actually seeing the distribution of the
clockevents_lock acquisition?


Thanks,

Nicolai



[1] https://nicst.de/cev-freqadjust/adjust_all_freqs-function_graph_hist.png

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | linux.kernel

csiph-web