Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1334722

[PATCHv2 4/4] slub: Relax CMPXCHG consistency restrictions

Show all headers | View raw

When debug options are enabled, cmpxchg on the page is disabled. This is
because the page must be locked to ensure there are no false positives
when performing consistency checks. Some debug options such as poisoning
and red zoning only act on the object itself. There is no need to
protect other CPUs from modification on only the object. Allow cmpxchg
to happen with poisoning and red zoning are set on a slab.

Credit to Mathias Krause for the original work which inspired this series

Signed-off-by: Laura Abbott <labbott@fedoraproject.org>
---
 mm/slub.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/mm/slub.c b/mm/slub.c
index 01606ff..0323e53 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -164,6 +164,14 @@ static inline bool kmem_cache_has_cpu_partial(struct kmem_cache *s)
 				SLAB_POISON | SLAB_STORE_USER)
 
 /*
+ * These debug flags cannot use CMPXCHG because there might be consistency
+ * issues when checking or reading debug information
+ */
+#define SLAB_NO_CMPXCHG (SLAB_CONSISTENCY_CHECKS | SLAB_STORE_USER | \
+				SLAB_TRACE)
+
+
+/*
  * Debugging flags that require metadata to be stored in the slab.  These get
  * disabled when slub_debug=O is used and a cache's min order increases with
  * metadata.
@@ -3377,7 +3385,7 @@ static int kmem_cache_open(struct kmem_cache *s, unsigned long flags)
 
 #if defined(CONFIG_HAVE_CMPXCHG_DOUBLE) && \
     defined(CONFIG_HAVE_ALIGNED_STRUCT_PAGE)
-	if (system_has_cmpxchg_double() && (s->flags & SLAB_DEBUG_FLAGS) == 0)
+	if (system_has_cmpxchg_double() && (s->flags & SLAB_NO_CMPXCHG) == 0)
 		/* Enable fast mode */
 		s->flags |= __CMPXCHG_DOUBLE;
 #endif
@@ -4889,7 +4897,6 @@ static ssize_t red_zone_store(struct kmem_cache *s,
 
 	s->flags &= ~SLAB_RED_ZONE;
 	if (buf[0] == '1') {
-		s->flags &= ~__CMPXCHG_DOUBLE;
 		s->flags |= SLAB_RED_ZONE;
 	}
 	calculate_sizes(s, -1);
@@ -4910,7 +4917,6 @@ static ssize_t poison_store(struct kmem_cache *s,
 
 	s->flags &= ~SLAB_POISON;
 	if (buf[0] == '1') {
-		s->flags &= ~__CMPXCHG_DOUBLE;
 		s->flags |= SLAB_POISON;
 	}
 	calculate_sizes(s, -1);
-- 
2.5.0

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCHv2 0/4] Improve performance for SLAB_POISON Laura Abbott <labbott@fedoraproject.org> - 2016-02-15 19:50 +0100
  [PATCHv2 2/4] slub: Fix/clean free_debug_processing return paths Laura Abbott <labbott@fedoraproject.org> - 2016-02-15 19:50 +0100
  [PATCHv2 4/4] slub: Relax CMPXCHG consistency restrictions Laura Abbott <labbott@fedoraproject.org> - 2016-02-15 19:50 +0100
  [PATCHv2 3/4] slub: Convert SLAB_DEBUG_FREE to SLAB_CONSISTENCY_CHECKS Laura Abbott <labbott@fedoraproject.org> - 2016-02-15 19:50 +0100
  [PATCHv2 1/4] slub: Drop lock at the end of free_debug_processing Laura Abbott <labbott@fedoraproject.org> - 2016-02-15 19:50 +0100
    Re: [PATCHv2 1/4] slub: Drop lock at the end of  free_debug_processing Christoph Lameter <cl@linux.com> - 2016-02-16 17:30 +0100
  Re: [PATCHv2 0/4] Improve performance for SLAB_POISON Joonsoo Kim <js1304@gmail.com> - 2016-02-18 09:40 +0100

csiph-web