Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1164473 > unrolled thread
| Started by | Andy Lutomirski <luto@kernel.org> |
|---|---|
| First post | 2015-06-13 02:00 +0200 |
| Last post | 2015-06-13 02:00 +0200 |
| Articles | 3 — 1 participant |
Back to article view | Back to linux.kernel
[PATCH v2 00/17] x86/tsc: Clean up rdtsc helpers Andy Lutomirski <luto@kernel.org> - 2015-06-13 02:00 +0200
[PATCH v2 03/17] x86/tsc/paravirt: Remove the read_tsc and read_tscp paravirt hooks Andy Lutomirski <luto@kernel.org> - 2015-06-13 02:00 +0200
[PATCH v2 01/17] x86/tsc: Inline native_read_tsc and remove __native_read_tsc Andy Lutomirski <luto@kernel.org> - 2015-06-13 02:00 +0200
| From | Andy Lutomirski <luto@kernel.org> |
|---|---|
| Date | 2015-06-13 02:00 +0200 |
| Subject | [PATCH v2 00/17] x86/tsc: Clean up rdtsc helpers |
| Message-ID | <pAGRX-44S-5@gated-at.bofh.it> |
My sincere apologies for the spam. I send an unholy mixture of the
real patch set and an old poorly split-up patch set, and the result
is incomprehensible. Here's what I meant to send.
After the some recent threads about rdtsc barriers, I remembered
that our RDTSC wrappers are a big mess. Let's clean it up.
Currently we have rdtscl, rdtscll, native_read_tsc,
paravirt_read_tsc, and rdtsc_barrier. For people who haven't
noticed rdtsc_barrier and who haven't carefully read the docs,
there's no indication that all of the other accessors have a giant
ordering gotcha. The macro forms are ugly, and the paravirt
implementation is completely pointless.
rdtscl is particularly awful. It reads the low bits. There are no
performance critical users of just the low bits anywhere in the
kernel.
Clean it up. After this patch set, there are exactly three
functions. rdtsc_unordered() is a function that does a raw RDTSC
and returns a 64-bit number. rdtsc_ordered() is a function that
does a properly ordered RDTSC for general-purpose use.
barrier_before_rdtsc() is exactly what it sounds like.
Andy Lutomirski (17):
x86/tsc: Inline native_read_tsc and remove __native_read_tsc
x86/msr/kvm: Remove vget_cycles()
x86/tsc/paravirt: Remove the read_tsc and read_tscp paravirt hooks
x86/tsc: Replace rdtscll with native_read_tsc
x86/tsc: Remove the rdtscp and rdtscpll macros
x86/tsc: Use the full 64-bit tsc in tsc_delay
x86/cpu/amd: Use the full 64-bit TSC to detect the 2.6.2 bug
baycom_epp: Replace rdtscl() with native_read_tsc()
staging/lirc_serial: Remove TSC-based timing
input/joystick/analog: Switch from rdtscl() to native_read_tsc()
drivers/input/gameport: Replace rdtscl() with native_read_tsc()
x86/tsc: Remove rdtscl()
x86/tsc: Rename native_read_tsc() to rdtsc_unordered()
x86/tsc: Move rdtsc_barrier() and rename it to barrier_before_rdtsc()
x86: Add rdtsc_ordered() and use it in trivial call sites
x86/tsc: Use rdtsc_ordered() in check_tsc_warp() and drop extra
barriers
x86/tsc: In read_tsc, use rdtsc_ordered() instead of get_cycles()
arch/x86/boot/compressed/aslr.c | 2 +-
arch/x86/entry/vdso/vclock_gettime.c | 16 +-----
arch/x86/include/asm/barrier.h | 11 ----
arch/x86/include/asm/msr.h | 63 +++++++++++++++-------
arch/x86/include/asm/paravirt.h | 34 ------------
arch/x86/include/asm/paravirt_types.h | 2 -
arch/x86/include/asm/pvclock.h | 8 +--
arch/x86/include/asm/stackprotector.h | 2 +-
arch/x86/include/asm/tsc.h | 18 +------
arch/x86/kernel/apb_timer.c | 8 +--
arch/x86/kernel/apic/apic.c | 8 +--
arch/x86/kernel/cpu/amd.c | 6 +--
arch/x86/kernel/cpu/mcheck/mce.c | 4 +-
arch/x86/kernel/espfix_64.c | 2 +-
arch/x86/kernel/hpet.c | 4 +-
arch/x86/kernel/paravirt.c | 2 -
arch/x86/kernel/paravirt_patch_32.c | 2 -
arch/x86/kernel/trace_clock.c | 7 +--
arch/x86/kernel/tsc.c | 12 ++---
arch/x86/kernel/tsc_sync.c | 14 +++--
arch/x86/kvm/lapic.c | 4 +-
arch/x86/kvm/svm.c | 4 +-
arch/x86/kvm/vmx.c | 4 +-
arch/x86/kvm/x86.c | 26 +++------
arch/x86/lib/delay.c | 13 ++---
arch/x86/um/asm/barrier.h | 4 +-
arch/x86/xen/enlighten.c | 3 --
drivers/input/gameport/gameport.c | 4 +-
drivers/input/joystick/analog.c | 4 +-
drivers/net/hamradio/baycom_epp.c | 2 +-
drivers/staging/media/lirc/lirc_serial.c | 63 ++--------------------
drivers/thermal/intel_powerclamp.c | 4 +-
.../power/cpupower/debug/kernel/cpufreq-test_tsc.c | 4 +-
33 files changed, 112 insertions(+), 252 deletions(-)
--
2.4.2
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [next] | [standalone]
| From | Andy Lutomirski <luto@kernel.org> |
|---|---|
| Date | 2015-06-13 02:00 +0200 |
| Subject | [PATCH v2 03/17] x86/tsc/paravirt: Remove the read_tsc and read_tscp paravirt hooks |
| Message-ID | <pAH1D-4gk-9@gated-at.bofh.it> |
| In reply to | #1164473 |
We've had read_tsc and read_tscp paravirt hooks since the very
beginning of paravirt, i.e., d3561b7fa0fb ("[PATCH] paravirt: header
and stubs for paravirtualisation"). AFAICT the only paravirt guest
implementation that ever replaced these calls was vmware, and it's
gone. Arguably even vmware shouldn't have hooked rdtsc -- we fully
support systems that don't have a TSC at all, so there's no point
for a paravirt implementation to pretend that we have a TSC but to
replace it.
I also doubt that these hooks actually worked. Calls to rdtscl and
rdtscll, which respected the hooks, were used seemingly
interchangeably with native_read_tsc, which did not.
Just remove them. If anyone ever needs them again, they can try
to make a case for why they need them.
Before, on a paravirt config:
text data bss dec hex filename
13426505 1827056 14508032 29761593 1c62039 vmlinux
After:
text data bss dec hex filename
13426617 1827056 14508032 29761705 1c620a9 vmlinux
Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
arch/x86/include/asm/msr.h | 16 ++++++++--------
arch/x86/include/asm/paravirt.h | 34 ----------------------------------
arch/x86/include/asm/paravirt_types.h | 2 --
arch/x86/kernel/paravirt.c | 2 --
arch/x86/kernel/paravirt_patch_32.c | 2 --
arch/x86/xen/enlighten.c | 3 ---
6 files changed, 8 insertions(+), 51 deletions(-)
diff --git a/arch/x86/include/asm/msr.h b/arch/x86/include/asm/msr.h
index 88711470af7f..d1afac7df484 100644
--- a/arch/x86/include/asm/msr.h
+++ b/arch/x86/include/asm/msr.h
@@ -178,12 +178,6 @@ static inline int rdmsrl_safe(unsigned msr, unsigned long long *p)
return err;
}
-#define rdtscl(low) \
- ((low) = (u32)native_read_tsc())
-
-#define rdtscll(val) \
- ((val) = native_read_tsc())
-
#define rdpmc(counter, low, high) \
do { \
u64 _l = native_read_pmc((counter)); \
@@ -193,6 +187,14 @@ do { \
#define rdpmcl(counter, val) ((val) = native_read_pmc(counter))
+#endif /* !CONFIG_PARAVIRT */
+
+#define rdtscl(low) \
+ ((low) = (u32)native_read_tsc())
+
+#define rdtscll(val) \
+ ((val) = native_read_tsc())
+
#define rdtscp(low, high, aux) \
do { \
unsigned long long _val = native_read_tscp(&(aux)); \
@@ -202,8 +204,6 @@ do { \
#define rdtscpll(val, aux) (val) = native_read_tscp(&(aux))
-#endif /* !CONFIG_PARAVIRT */
-
/*
* 64-bit version of wrmsr_safe():
*/
diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h
index d143bfad45d7..c2be0375bcad 100644
--- a/arch/x86/include/asm/paravirt.h
+++ b/arch/x86/include/asm/paravirt.h
@@ -174,19 +174,6 @@ static inline int rdmsrl_safe(unsigned msr, unsigned long long *p)
return err;
}
-static inline u64 paravirt_read_tsc(void)
-{
- return PVOP_CALL0(u64, pv_cpu_ops.read_tsc);
-}
-
-#define rdtscl(low) \
-do { \
- u64 _l = paravirt_read_tsc(); \
- low = (int)_l; \
-} while (0)
-
-#define rdtscll(val) (val = paravirt_read_tsc())
-
static inline unsigned long long paravirt_sched_clock(void)
{
return PVOP_CALL0(unsigned long long, pv_time_ops.sched_clock);
@@ -215,27 +202,6 @@ do { \
#define rdpmcl(counter, val) ((val) = paravirt_read_pmc(counter))
-static inline unsigned long long paravirt_rdtscp(unsigned int *aux)
-{
- return PVOP_CALL1(u64, pv_cpu_ops.read_tscp, aux);
-}
-
-#define rdtscp(low, high, aux) \
-do { \
- int __aux; \
- unsigned long __val = paravirt_rdtscp(&__aux); \
- (low) = (u32)__val; \
- (high) = (u32)(__val >> 32); \
- (aux) = __aux; \
-} while (0)
-
-#define rdtscpll(val, aux) \
-do { \
- unsigned long __aux; \
- val = paravirt_rdtscp(&__aux); \
- (aux) = __aux; \
-} while (0)
-
static inline void paravirt_alloc_ldt(struct desc_struct *ldt, unsigned entries)
{
PVOP_VCALL2(pv_cpu_ops.alloc_ldt, ldt, entries);
diff --git a/arch/x86/include/asm/paravirt_types.h b/arch/x86/include/asm/paravirt_types.h
index a6b8f9fadb06..ce029e4fa7c6 100644
--- a/arch/x86/include/asm/paravirt_types.h
+++ b/arch/x86/include/asm/paravirt_types.h
@@ -156,9 +156,7 @@ struct pv_cpu_ops {
u64 (*read_msr)(unsigned int msr, int *err);
int (*write_msr)(unsigned int msr, unsigned low, unsigned high);
- u64 (*read_tsc)(void);
u64 (*read_pmc)(int counter);
- unsigned long long (*read_tscp)(unsigned int *aux);
#ifdef CONFIG_X86_32
/*
diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c
index 58bcfb67c01f..f68e48f5f6c2 100644
--- a/arch/x86/kernel/paravirt.c
+++ b/arch/x86/kernel/paravirt.c
@@ -351,9 +351,7 @@ __visible struct pv_cpu_ops pv_cpu_ops = {
.wbinvd = native_wbinvd,
.read_msr = native_read_msr_safe,
.write_msr = native_write_msr_safe,
- .read_tsc = native_read_tsc,
.read_pmc = native_read_pmc,
- .read_tscp = native_read_tscp,
.load_tr_desc = native_load_tr_desc,
.set_ldt = native_set_ldt,
.load_gdt = native_load_gdt,
diff --git a/arch/x86/kernel/paravirt_patch_32.c b/arch/x86/kernel/paravirt_patch_32.c
index e1b013696dde..c89f50a76e97 100644
--- a/arch/x86/kernel/paravirt_patch_32.c
+++ b/arch/x86/kernel/paravirt_patch_32.c
@@ -10,7 +10,6 @@ DEF_NATIVE(pv_mmu_ops, read_cr2, "mov %cr2, %eax");
DEF_NATIVE(pv_mmu_ops, write_cr3, "mov %eax, %cr3");
DEF_NATIVE(pv_mmu_ops, read_cr3, "mov %cr3, %eax");
DEF_NATIVE(pv_cpu_ops, clts, "clts");
-DEF_NATIVE(pv_cpu_ops, read_tsc, "rdtsc");
#if defined(CONFIG_PARAVIRT_SPINLOCKS) && defined(CONFIG_QUEUED_SPINLOCKS)
DEF_NATIVE(pv_lock_ops, queued_spin_unlock, "movb $0, (%eax)");
@@ -52,7 +51,6 @@ unsigned native_patch(u8 type, u16 clobbers, void *ibuf,
PATCH_SITE(pv_mmu_ops, read_cr3);
PATCH_SITE(pv_mmu_ops, write_cr3);
PATCH_SITE(pv_cpu_ops, clts);
- PATCH_SITE(pv_cpu_ops, read_tsc);
#if defined(CONFIG_PARAVIRT_SPINLOCKS) && defined(CONFIG_QUEUED_SPINLOCKS)
case PARAVIRT_PATCH(pv_lock_ops.queued_spin_unlock):
if (pv_is_native_spin_unlock()) {
diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
index 0b95c9b8283f..32136bfca43f 100644
--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -1175,11 +1175,8 @@ static const struct pv_cpu_ops xen_cpu_ops __initconst = {
.read_msr = xen_read_msr_safe,
.write_msr = xen_write_msr_safe,
- .read_tsc = native_read_tsc,
.read_pmc = native_read_pmc,
- .read_tscp = native_read_tscp,
-
.iret = xen_iret,
#ifdef CONFIG_X86_64
.usergs_sysret32 = xen_sysret32,
--
2.4.2
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Andy Lutomirski <luto@kernel.org> |
|---|---|
| Date | 2015-06-13 02:00 +0200 |
| Subject | [PATCH v2 01/17] x86/tsc: Inline native_read_tsc and remove __native_read_tsc |
| Message-ID | <pAH1E-4gk-15@gated-at.bofh.it> |
| In reply to | #1164473 |
In cdc7957d1954 ("x86: move native_read_tsc() offline"),
native_read_tsc was moved out of line, presumably for some
now-obsolete vDSO-related reason. Undo it.
The entire rdtsc, shl, or sequence is only 11 bytes, and calls via
rdtscl and similar helpers were already inlined.
Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
arch/x86/entry/vdso/vclock_gettime.c | 2 +-
arch/x86/include/asm/msr.h | 8 +++-----
arch/x86/include/asm/pvclock.h | 2 +-
arch/x86/include/asm/stackprotector.h | 2 +-
arch/x86/include/asm/tsc.h | 2 +-
arch/x86/kernel/apb_timer.c | 4 ++--
arch/x86/kernel/tsc.c | 6 ------
7 files changed, 9 insertions(+), 17 deletions(-)
diff --git a/arch/x86/entry/vdso/vclock_gettime.c b/arch/x86/entry/vdso/vclock_gettime.c
index 9793322751e0..972b488ac16a 100644
--- a/arch/x86/entry/vdso/vclock_gettime.c
+++ b/arch/x86/entry/vdso/vclock_gettime.c
@@ -186,7 +186,7 @@ notrace static cycle_t vread_tsc(void)
* but no one has ever seen it happen.
*/
rdtsc_barrier();
- ret = (cycle_t)__native_read_tsc();
+ ret = (cycle_t)native_read_tsc();
last = gtod->cycle_last;
diff --git a/arch/x86/include/asm/msr.h b/arch/x86/include/asm/msr.h
index e6a707eb5081..88711470af7f 100644
--- a/arch/x86/include/asm/msr.h
+++ b/arch/x86/include/asm/msr.h
@@ -106,12 +106,10 @@ notrace static inline int native_write_msr_safe(unsigned int msr,
return err;
}
-extern unsigned long long native_read_tsc(void);
-
extern int rdmsr_safe_regs(u32 regs[8]);
extern int wrmsr_safe_regs(u32 regs[8]);
-static __always_inline unsigned long long __native_read_tsc(void)
+static __always_inline unsigned long long native_read_tsc(void)
{
DECLARE_ARGS(val, low, high);
@@ -181,10 +179,10 @@ static inline int rdmsrl_safe(unsigned msr, unsigned long long *p)
}
#define rdtscl(low) \
- ((low) = (u32)__native_read_tsc())
+ ((low) = (u32)native_read_tsc())
#define rdtscll(val) \
- ((val) = __native_read_tsc())
+ ((val) = native_read_tsc())
#define rdpmc(counter, low, high) \
do { \
diff --git a/arch/x86/include/asm/pvclock.h b/arch/x86/include/asm/pvclock.h
index d6b078e9fa28..71bd485c2986 100644
--- a/arch/x86/include/asm/pvclock.h
+++ b/arch/x86/include/asm/pvclock.h
@@ -62,7 +62,7 @@ static inline u64 pvclock_scale_delta(u64 delta, u32 mul_frac, int shift)
static __always_inline
u64 pvclock_get_nsec_offset(const struct pvclock_vcpu_time_info *src)
{
- u64 delta = __native_read_tsc() - src->tsc_timestamp;
+ u64 delta = native_read_tsc() - src->tsc_timestamp;
return pvclock_scale_delta(delta, src->tsc_to_system_mul,
src->tsc_shift);
}
diff --git a/arch/x86/include/asm/stackprotector.h b/arch/x86/include/asm/stackprotector.h
index c2e00bb2a136..bc5fa2af112e 100644
--- a/arch/x86/include/asm/stackprotector.h
+++ b/arch/x86/include/asm/stackprotector.h
@@ -72,7 +72,7 @@ static __always_inline void boot_init_stack_canary(void)
* on during the bootup the random pool has true entropy too.
*/
get_random_bytes(&canary, sizeof(canary));
- tsc = __native_read_tsc();
+ tsc = native_read_tsc();
canary += tsc + (tsc << 32UL);
current->stack_canary = canary;
diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h
index 94605c0e9cee..fd11128faf25 100644
--- a/arch/x86/include/asm/tsc.h
+++ b/arch/x86/include/asm/tsc.h
@@ -42,7 +42,7 @@ static __always_inline cycles_t vget_cycles(void)
if (!cpu_has_tsc)
return 0;
#endif
- return (cycles_t)__native_read_tsc();
+ return (cycles_t)native_read_tsc();
}
extern void tsc_init(void);
diff --git a/arch/x86/kernel/apb_timer.c b/arch/x86/kernel/apb_timer.c
index ede92c3364d3..9fe111cc50f8 100644
--- a/arch/x86/kernel/apb_timer.c
+++ b/arch/x86/kernel/apb_timer.c
@@ -390,13 +390,13 @@ unsigned long apbt_quick_calibrate(void)
old = dw_apb_clocksource_read(clocksource_apbt);
old += loop;
- t1 = __native_read_tsc();
+ t1 = native_read_tsc();
do {
new = dw_apb_clocksource_read(clocksource_apbt);
} while (new < old);
- t2 = __native_read_tsc();
+ t2 = native_read_tsc();
shift = 5;
if (unlikely(loop >> shift == 0)) {
diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
index 505449700e0c..e7710cd7ba00 100644
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -308,12 +308,6 @@ unsigned long long
sched_clock(void) __attribute__((alias("native_sched_clock")));
#endif
-unsigned long long native_read_tsc(void)
-{
- return __native_read_tsc();
-}
-EXPORT_SYMBOL(native_read_tsc);
-
int check_tsc_unstable(void)
{
return tsc_unstable;
--
2.4.2
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [standalone]
Back to top | Article view | linux.kernel
csiph-web