Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1229921

[PATCH 09/11] x86/mm: Make pgd_alloc()/pgd_free() lockless

Show all headers | View raw

The fork()/exit() code uses pgd_alloc()/pgd_free() to allocate/deallocate
the PGD, with platform specific code setting up kernel pagetables.

The x86 code uses a global pgd_list with an associated lock to update
all PGDs of all tasks in the system synchronously.

The lock is still kept to synchronize updates to all PGDs in the system,
but all users of the list have been migrated to use the task list.

So we can remove the pgd_list addition/removal from this code.

The new PGD is private while constructed, so it needs no extra
locking.

Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Waiman Long <Waiman.Long@hp.com>
Cc: linux-mm@kvack.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 arch/x86/mm/pgtable.c | 27 +++------------------------
 1 file changed, 3 insertions(+), 24 deletions(-)

diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c
index c7038b6e51bf..8a42d54f44ba 100644
--- a/arch/x86/mm/pgtable.c
+++ b/arch/x86/mm/pgtable.c
@@ -125,22 +125,6 @@ static void pgd_ctor(struct mm_struct *mm, pgd_t *pgd)
 				swapper_pg_dir + KERNEL_PGD_BOUNDARY,
 				KERNEL_PGD_PTRS);
 	}
-
-	/* list required to sync kernel mapping updates */
-	if (!SHARED_KERNEL_PMD) {
-		pgd_set_mm(pgd, mm);
-		pgd_list_add(pgd);
-	}
-}
-
-static void pgd_dtor(pgd_t *pgd)
-{
-	if (SHARED_KERNEL_PMD)
-		return;
-
-	spin_lock(&pgd_lock);
-	pgd_list_del(pgd);
-	spin_unlock(&pgd_lock);
 }
 
 /*
@@ -370,17 +354,13 @@ pgd_t *pgd_alloc(struct mm_struct *mm)
 		goto out_free_pmds;
 
 	/*
-	 * Make sure that pre-populating the pmds is atomic with
-	 * respect to anything walking the pgd_list, so that they
-	 * never see a partially populated pgd.
+	 * No locking is needed here, as the PGD is still private,
+	 * so no code walking the task list and looking at mm->pgd
+	 * will be able to see it before it's fully constructed:
 	 */
-	spin_lock(&pgd_lock);
-
 	pgd_ctor(mm, pgd);
 	pgd_prepopulate_pmd(mm, pgd, pmds);
 
-	spin_unlock(&pgd_lock);
-
 	return pgd;
 
 out_free_pmds:
@@ -453,7 +433,6 @@ void arch_pgd_init_late(struct mm_struct *mm)
 void pgd_free(struct mm_struct *mm, pgd_t *pgd)
 {
 	pgd_mop_up_pmds(mm, pgd);
-	pgd_dtor(pgd);
 	paravirt_pgd_free(mm, pgd);
 	_pgd_free(pgd);
 }
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH 00/11] x86/mm: Implement lockless pgd_alloc()/pgd_free() Ingo Molnar <mingo@kernel.org> - 2015-09-22 08:30 +0200
  [PATCH 07/11] x86/mm: Remove pgd_list use from vmalloc_sync_all() Ingo Molnar <mingo@kernel.org> - 2015-09-22 08:30 +0200
    Re: [PATCH 07/11] x86/mm: Remove pgd_list use from vmalloc_sync_all() Linus Torvalds <torvalds@linux-foundation.org> - 2015-09-22 20:00 +0200
  [PATCH 06/11] x86/virt/guest/xen: Remove use of pgd_list from the Xen guest code Ingo Molnar <mingo@kernel.org> - 2015-09-22 08:30 +0200
    Re: [PATCH 06/11] x86/virt/guest/xen: Remove use of pgd_list from the  Xen guest code Linus Torvalds <torvalds@linux-foundation.org> - 2015-09-22 20:00 +0200
      Re: [PATCH 06/11] x86/virt/guest/xen: Remove use of pgd_list from  the Xen guest code Ingo Molnar <mingo@kernel.org> - 2015-09-29 10:50 +0200
  [PATCH 10/11] x86/mm: Remove pgd_list leftovers Ingo Molnar <mingo@kernel.org> - 2015-09-22 08:30 +0200
  [PATCH 08/11] x86/mm/pat/32: Remove pgd_list use from the PAT code Ingo Molnar <mingo@kernel.org> - 2015-09-22 08:30 +0200
  [PATCH 09/11] x86/mm: Make pgd_alloc()/pgd_free() lockless Ingo Molnar <mingo@kernel.org> - 2015-09-22 08:30 +0200
  [PATCH 02/11] x86/mm/hotplug: Remove pgd_list use from the memory hotplug code Ingo Molnar <mingo@kernel.org> - 2015-09-22 08:30 +0200
    Re: [PATCH 02/11] x86/mm/hotplug: Remove pgd_list use from the memory  hotplug code Linus Torvalds <torvalds@linux-foundation.org> - 2015-09-22 19:50 +0200
      Re: [PATCH 02/11] x86/mm/hotplug: Remove pgd_list use from the  memory hotplug code Oleg Nesterov <oleg@redhat.com> - 2015-09-23 13:50 +0200
        Re: [PATCH 02/11] x86/mm/hotplug: Remove pgd_list use from the  memory hotplug code Ingo Molnar <mingo@kernel.org> - 2015-09-29 10:50 +0200
          Re: [PATCH 02/11] x86/mm/hotplug: Remove pgd_list use from the  memory hotplug code Oleg Nesterov <oleg@redhat.com> - 2015-09-29 19:00 +0200
  [PATCH 03/11] x86/mm/hotplug: Don't remove PGD entries in remove_pagetable() Ingo Molnar <mingo@kernel.org> - 2015-09-22 08:30 +0200
  [PATCH 01/11] x86/mm/pat: Don't free PGD entries on memory unmap Ingo Molnar <mingo@kernel.org> - 2015-09-22 08:30 +0200
    Re: [PATCH 01/11] x86/mm/pat: Don't free PGD entries on memory unmap Linus Torvalds <torvalds@linux-foundation.org> - 2015-09-22 19:50 +0200
      Re: [PATCH 01/11] x86/mm/pat: Don't free PGD entries on memory unmap Andy Lutomirski <luto@amacapital.net> - 2015-09-22 20:10 +0200
  [PATCH 04/11] x86/mm/hotplug: Simplify sync_global_pgds() Ingo Molnar <mingo@kernel.org> - 2015-09-22 08:30 +0200

csiph-web