Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1375989

[PATCH 3.12 05/98] kernel: Provide READ_ONCE and ASSIGN_ONCE

Show all headers | View raw

From: Christian Borntraeger <borntraeger@de.ibm.com>

3.12-stable review patch.  If anyone has any objections, please let me know.

===============

commit 230fa253df6352af12ad0a16128760b5cb3f92df upstream.

ACCESS_ONCE does not work reliably on non-scalar types. For
example gcc 4.6 and 4.7 might remove the volatile tag for such
accesses during the SRA (scalar replacement of aggregates) step
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58145)

Let's provide READ_ONCE/ASSIGN_ONCE that will do all accesses via
scalar types as suggested by Linus Torvalds. Accesses larger than
the machines word size cannot be guaranteed to be atomic. These
macros will use memcpy and emit a build warning.

Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: Jiri Slaby <jslaby@suse.cz>
---
 include/linux/compiler.h | 74 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 74 insertions(+)

diff --git a/include/linux/compiler.h b/include/linux/compiler.h
index 19a199414bd0..237063adbe1b 100644
--- a/include/linux/compiler.h
+++ b/include/linux/compiler.h
@@ -179,6 +179,80 @@ void ftrace_likely_update(struct ftrace_branch_data *f, int val, int expect);
 # define __UNIQUE_ID(prefix) __PASTE(__PASTE(__UNIQUE_ID_, prefix), __LINE__)
 #endif
 
+#include <uapi/linux/types.h>
+
+static __always_inline void data_access_exceeds_word_size(void)
+#ifdef __compiletime_warning
+__compiletime_warning("data access exceeds word size and won't be atomic")
+#endif
+;
+
+static __always_inline void data_access_exceeds_word_size(void)
+{
+}
+
+static __always_inline void __read_once_size(volatile void *p, void *res, int size)
+{
+	switch (size) {
+	case 1: *(__u8 *)res = *(volatile __u8 *)p; break;
+	case 2: *(__u16 *)res = *(volatile __u16 *)p; break;
+	case 4: *(__u32 *)res = *(volatile __u32 *)p; break;
+#ifdef CONFIG_64BIT
+	case 8: *(__u64 *)res = *(volatile __u64 *)p; break;
+#endif
+	default:
+		barrier();
+		__builtin_memcpy((void *)res, (const void *)p, size);
+		data_access_exceeds_word_size();
+		barrier();
+	}
+}
+
+static __always_inline void __assign_once_size(volatile void *p, void *res, int size)
+{
+	switch (size) {
+	case 1: *(volatile __u8 *)p = *(__u8 *)res; break;
+	case 2: *(volatile __u16 *)p = *(__u16 *)res; break;
+	case 4: *(volatile __u32 *)p = *(__u32 *)res; break;
+#ifdef CONFIG_64BIT
+	case 8: *(volatile __u64 *)p = *(__u64 *)res; break;
+#endif
+	default:
+		barrier();
+		__builtin_memcpy((void *)p, (const void *)res, size);
+		data_access_exceeds_word_size();
+		barrier();
+	}
+}
+
+/*
+ * Prevent the compiler from merging or refetching reads or writes. The
+ * compiler is also forbidden from reordering successive instances of
+ * READ_ONCE, ASSIGN_ONCE and ACCESS_ONCE (see below), but only when the
+ * compiler is aware of some particular ordering.  One way to make the
+ * compiler aware of ordering is to put the two invocations of READ_ONCE,
+ * ASSIGN_ONCE or ACCESS_ONCE() in different C statements.
+ *
+ * In contrast to ACCESS_ONCE these two macros will also work on aggregate
+ * data types like structs or unions. If the size of the accessed data
+ * type exceeds the word size of the machine (e.g., 32 bits or 64 bits)
+ * READ_ONCE() and ASSIGN_ONCE()  will fall back to memcpy and print a
+ * compile-time warning.
+ *
+ * Their two major use cases are: (1) Mediating communication between
+ * process-level code and irq/NMI handlers, all running on the same CPU,
+ * and (2) Ensuring that the compiler does not  fold, spindle, or otherwise
+ * mutilate accesses that either do not require ordering or that interact
+ * with an explicit memory barrier or atomic instruction that provides the
+ * required ordering.
+ */
+
+#define READ_ONCE(x) \
+	({ typeof(x) __val; __read_once_size(&x, &__val, sizeof(__val)); __val; })
+
+#define ASSIGN_ONCE(val, x) \
+	({ typeof(x) __val; __val = val; __assign_once_size(&x, &__val, sizeof(__val)); __val; })
+
 #endif /* __KERNEL__ */
 
 #endif /* __ASSEMBLY__ */
-- 
2.8.1

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH 3.12 00/98] 3.12.58-stable review Jiri Slaby <jslaby@suse.cz> - 2016-04-11 15:30 +0200
  Re: [PATCH 3.12 00/98] 3.12.58-stable review Guenter Roeck <linux@roeck-us.net> - 2016-04-11 15:40 +0200
    Re: [PATCH 3.12 00/98] 3.12.58-stable review Jiri Slaby <jslaby@suse.cz> - 2016-04-14 10:30 +0200
  [PATCH 3.12 01/98] ipr: Fix out-of-bounds null overwrite Jiri Slaby <jslaby@suse.cz> - 2016-04-11 16:10 +0200
    [PATCH 3.12 18/98] ALSA: rawmidi: Make snd_rawmidi_transmit() race-free Jiri Slaby <jslaby@suse.cz> - 2016-04-11 16:10 +0200
    [PATCH 3.12 23/98] KVM: SVM: add rdmsr support for AMD event registers Jiri Slaby <jslaby@suse.cz> - 2016-04-11 16:10 +0200
    [PATCH 3.12 08/98] xen-netback: use RING_COPY_REQUEST() throughout Jiri Slaby <jslaby@suse.cz> - 2016-04-11 16:10 +0200
    [PATCH 3.12 07/98] xen-netback: don't use last request to determine minimum Tx credit Jiri Slaby <jslaby@suse.cz> - 2016-04-11 16:10 +0200
    [PATCH 3.12 03/98] ipv4: Don't do expensive useless work during inetdev destroy. Jiri Slaby <jslaby@suse.cz> - 2016-04-11 16:10 +0200
    [PATCH 3.12 27/98] perf, nmi: Fix unknown NMI warning Jiri Slaby <jslaby@suse.cz> - 2016-04-11 16:10 +0200
    [PATCH 3.12 14/98] sched: Fix race between task_group and sched_task_group Jiri Slaby <jslaby@suse.cz> - 2016-04-11 16:10 +0200
    [PATCH 3.12 20/98] memcg: do not hang on OOM when killed by userspace OOM access to memory reserves Jiri Slaby <jslaby@suse.cz> - 2016-04-11 16:10 +0200
    [PATCH 3.12 19/98] ALSA: seq: Fix leak of pool buffer at concurrent writes Jiri Slaby <jslaby@suse.cz> - 2016-04-11 16:10 +0200
    [PATCH 3.12 09/98] xen-blkback: only read request operation from shared ring once Jiri Slaby <jslaby@suse.cz> - 2016-04-11 16:10 +0200
    [PATCH 3.12 05/98] kernel: Provide READ_ONCE and ASSIGN_ONCE Jiri Slaby <jslaby@suse.cz> - 2016-04-11 16:10 +0200
      Re: [PATCH 3.12 05/98] kernel: Provide READ_ONCE and ASSIGN_ONCE Christian Borntraeger <borntraeger@de.ibm.com> - 2016-04-11 16:20 +0200
    [PATCH 3.12 06/98] xen: Add RING_COPY_REQUEST() Jiri Slaby <jslaby@suse.cz> - 2016-04-11 16:10 +0200
    [PATCH 3.12 17/98] drivers/base/memory.c: fix kernel warning during memory hotplug on ppc64 Jiri Slaby <jslaby@suse.cz> - 2016-04-11 16:10 +0200
    [PATCH 3.12 04/98] umount: Do not allow unmounting rootfs. Jiri Slaby <jslaby@suse.cz> - 2016-04-11 16:10 +0200
    [PATCH 3.12 10/98] xen-blkback: read from indirect descriptors only once Jiri Slaby <jslaby@suse.cz> - 2016-04-11 16:10 +0200
    [PATCH 3.12 02/98] ipr: Fix regression when loading firmware Jiri Slaby <jslaby@suse.cz> - 2016-04-11 16:10 +0200
    [PATCH 3.12 15/98] Btrfs: skip locking when searching commit root Jiri Slaby <jslaby@suse.cz> - 2016-04-11 16:10 +0200
  Re: [PATCH 3.12 00/98] 3.12.58-stable review shuahkh <shuahkh@osg.sisa.samsung.com> - 2016-04-11 19:30 +0200

csiph-web