Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1353353

[PATCH v12 23/29] HMM: new callback for copying memory from and to device memory v2.

From Jérôme Glisse <jglisse@redhat.com>
Newsgroups linux.kernel
Subject [PATCH v12 23/29] HMM: new callback for copying memory from and to device memory v2.
Date 2016-03-08 21:00 +0100
Message-ID <ravXs-5uX-5@gated-at.bofh.it> (permalink)
References <ravNL-5ru-3@gated-at.bofh.it>
Organization linux.* mail to news gateway

Show all headers | View raw

From: Jerome Glisse <jglisse@redhat.com>

This patch only adds the new callback device driver must implement
to copy memory from and to device memory.

Changed since v1:
  - Pass down the vma to the copy function.

Signed-off-by: Jérôme Glisse <jglisse@redhat.com>
Signed-off-by: Sherry Cheung <SCheung@nvidia.com>
Signed-off-by: Subhash Gutti <sgutti@nvidia.com>
Signed-off-by: Mark Hairgrove <mhairgrove@nvidia.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Signed-off-by: Jatin Kumar <jakumar@nvidia.com>
---
 include/linux/hmm.h | 105 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 mm/hmm.c            |   2 +
 2 files changed, 107 insertions(+)

diff --git a/include/linux/hmm.h b/include/linux/hmm.h
index 7c66513..9fbfc07 100644
--- a/include/linux/hmm.h
+++ b/include/linux/hmm.h
@@ -65,6 +65,8 @@ enum hmm_etype {
 	HMM_DEVICE_RFAULT,
 	HMM_DEVICE_WFAULT,
 	HMM_WRITE_PROTECT,
+	HMM_COPY_FROM_DEVICE,
+	HMM_COPY_TO_DEVICE,
 };
 
 /* struct hmm_event - memory event information.
@@ -170,6 +172,109 @@ struct hmm_device_ops {
 	 */
 	int (*update)(struct hmm_mirror *mirror,
 		      struct hmm_event *event);
+
+	/* copy_from_device() - copy from device memory to system memory.
+	 *
+	 * @mirror: The mirror that link process address space with the device.
+	 * @event: The event that triggered the copy.
+	 * @dst: Array containing hmm_pte of destination memory.
+	 * @start: Start address of the range (sub-range of event) to copy.
+	 * @end: End address of the range (sub-range of event) to copy.
+	 * Returns: 0 on success, error code otherwise {-ENOMEM, -EIO}.
+	 *
+	 * Called when migrating memory from device memory to system memory.
+	 * The dst array contains valid DMA address for the device of the page
+	 * to copy to (or pfn of page if hmm_device.device == NULL).
+	 *
+	 * If event.etype == HMM_FORK then device driver only need to schedule
+	 * a copy to the system pages given in the dst hmm_pte array. Do not
+	 * update the device page, and do not pause/stop the device threads
+	 * that are using this address space. Just copy memory.
+	 *
+	 * If event.type == HMM_COPY_FROM_DEVICE then device driver must first
+	 * write protect the range then schedule the copy, then update its page
+	 * table to use the new system memory given the dst array. Some device
+	 * can perform all this in an atomic fashion from device point of view.
+	 * The device driver must also free the device memory once the copy is
+	 * done.
+	 *
+	 * Device driver must not fail lightly, any failure result in device
+	 * process being kill and CPU page table set to HWPOISON entry.
+	 *
+	 * Note that device driver must clear the valid bit of the dst entry it
+	 * failed to copy.
+	 *
+	 * On failure the mirror will be kill by HMM which will do a HMM_MUNMAP
+	 * invalidation of all the memory when this happen the device driver
+	 * can free the device memory.
+	 *
+	 * Note also that there can be hole in the range being copied ie some
+	 * entry of dst array will not have the valid bit set, device driver
+	 * must simply ignore non valid entry.
+	 *
+	 * Finaly device driver must set the dirty bit for each page that was
+	 * modified since it was copied inside the device memory. This must be
+	 * conservative ie if device can not determine that with certainty then
+	 * it must set the dirty bit unconditionally.
+	 *
+	 * Return 0 on success, error value otherwise :
+	 * -ENOMEM Not enough memory for performing the operation.
+	 * -EIO    Some input/output error with the device.
+	 *
+	 * All other return value trigger warning and are transformed to -EIO.
+	 */
+	int (*copy_from_device)(struct hmm_mirror *mirror,
+				const struct hmm_event *event,
+				dma_addr_t *dst,
+				unsigned long start,
+				unsigned long end);
+
+	/* copy_to_device() - copy to device memory from system memory.
+	 *
+	 * @mirror: The mirror that link process address space with the device.
+	 * @event: The event that triggered the copy.
+	 * @vma: The vma corresponding to the range.
+	 * @dst: Array containing hmm_pte of destination memory.
+	 * @start: Start address of the range (sub-range of event) to copy.
+	 * @end: End address of the range (sub-range of event) to copy.
+	 * Returns: 0 on success, error code otherwise {-ENOMEM, -EIO}.
+	 *
+	 * Called when migrating memory from system memory to device memory.
+	 * The dst array is empty, all of its entry are equal to zero. Device
+	 * driver must allocate the device memory and populate each entry using
+	 * hmm_pte_from_device_pfn() only the valid device bit and hardware
+	 * specific bit will be preserve (write and dirty will be taken from
+	 * the original entry inside the mirror page table). It is advice to
+	 * set the device pfn to match the physical address of device memory
+	 * being use. The event.etype will be equals to HMM_COPY_TO_DEVICE.
+	 *
+	 * Device driver that can atomically copy a page and update its page
+	 * table entry to point to the device memory can do that. Partial
+	 * failure is allowed, entry that have not been migrated must have
+	 * the HMM_PTE_VALID_DEV bit clear inside the dst array. HMM will
+	 * update the CPU page table of failed entry to point back to the
+	 * system page.
+	 *
+	 * Note that device driver is responsible for allocating and freeing
+	 * the device memory and properly updating to dst array entry with
+	 * the allocated device memory.
+	 *
+	 * Return 0 on success, error value otherwise :
+	 * -ENOMEM Not enough memory for performing the operation.
+	 * -EIO    Some input/output error with the device.
+	 *
+	 * All other return value trigger warning and are transformed to -EIO.
+	 * Errors means that the migration is aborted. So in case of partial
+	 * failure if device do not want to fully abort it must return 0.
+	 * Device driver can update device page table only if it knows it will
+	 * not return failure.
+	 */
+	int (*copy_to_device)(struct hmm_mirror *mirror,
+			      const struct hmm_event *event,
+			      struct vm_area_struct *vma,
+			      dma_addr_t *dst,
+			      unsigned long start,
+			      unsigned long end);
 };
 
 
diff --git a/mm/hmm.c b/mm/hmm.c
index 9455443..d26abe4 100644
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@ -78,6 +78,8 @@ static inline int hmm_event_init(struct hmm_event *event,
 	switch (etype) {
 	case HMM_DEVICE_RFAULT:
 	case HMM_DEVICE_WFAULT:
+	case HMM_COPY_TO_DEVICE:
+	case HMM_COPY_FROM_DEVICE:
 		break;
 	case HMM_FORK:
 	case HMM_WRITE_PROTECT:
-- 
2.4.3

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

HMM (Heterogeneous Memory Management) Jérôme Glisse <jglisse@redhat.com> - 2016-03-08 20:50 +0100
  [PATCH v12 11/29] HMM: add discard range helper (to clear and free resources for a range). Jérôme Glisse <jglisse@redhat.com> - 2016-03-08 20:50 +0100
  [PATCH v12 19/29] HMM: add new HMM page table flag (select flag). Jérôme Glisse <jglisse@redhat.com> - 2016-03-08 20:50 +0100
  [PATCH v12 03/29] mmu_notifier: pass page pointer to mmu_notifier_invalidate_page() v2 Jérôme Glisse <jglisse@redhat.com> - 2016-03-08 20:50 +0100
  [PATCH v12 05/29] HMM: introduce heterogeneous memory management v5. Jérôme Glisse <jglisse@redhat.com> - 2016-03-08 20:50 +0100
  [PATCH v12 06/29] HMM: add HMM page table v4. Jérôme Glisse <jglisse@redhat.com> - 2016-03-08 20:50 +0100
  [PATCH v12 10/29] HMM: use CPU page table during invalidation. Jérôme Glisse <jglisse@redhat.com> - 2016-03-08 20:50 +0100
  [PATCH v12 29/29] HMM: add mirror fault support for system to device memory migration v3. Jérôme Glisse <jglisse@redhat.com> - 2016-03-08 20:50 +0100
  [PATCH v12 24/29] HMM: allow to get pointer to spinlock protecting a directory. Jérôme Glisse <jglisse@redhat.com> - 2016-03-08 20:50 +0100
  [PATCH v12 07/29] HMM: add per mirror page table v4. Jérôme Glisse <jglisse@redhat.com> - 2016-03-08 20:50 +0100
  [PATCH v12 15/29] HMM: add documentation explaining HMM internals and how to use it. Jérôme Glisse <jglisse@redhat.com> - 2016-03-08 20:50 +0100
  [PATCH v12 17/29] HMM: add special swap filetype for memory migrated to device v2. Jérôme Glisse <jglisse@redhat.com> - 2016-03-08 20:50 +0100
  [PATCH v12 21/29] HMM: mm add helper to update page table when migrating memory back v2. Jérôme Glisse <jglisse@redhat.com> - 2016-03-08 20:50 +0100
  [PATCH v12 23/29] HMM: new callback for copying memory from and to device memory v2. Jérôme Glisse <jglisse@redhat.com> - 2016-03-08 21:00 +0100
  [PATCH v12 13/29] HMM: DMA map memory on behalf of device driver v2. Jérôme Glisse <jglisse@redhat.com> - 2016-03-08 21:00 +0100
  [PATCH v12 14/29] HMM: Add support for hugetlb. Jérôme Glisse <jglisse@redhat.com> - 2016-03-08 21:00 +0100
  [PATCH v12 28/29] HMM: CPU page fault on migrated memory. Jérôme Glisse <jglisse@redhat.com> - 2016-03-08 21:00 +0100
  [PATCH v12 26/29] HMM: add helpers for migration back to system memory v3. Jérôme Glisse <jglisse@redhat.com> - 2016-03-08 21:00 +0100
  [PATCH v12 25/29] HMM: split DMA mapping function in two. Jérôme Glisse <jglisse@redhat.com> - 2016-03-08 21:00 +0100
  [PATCH v12 08/29] HMM: add device page fault support v6. Jérôme Glisse <jglisse@redhat.com> - 2016-03-08 21:00 +0100
  [PATCH v12 18/29] HMM: add new HMM page table flag (valid device memory). Jérôme Glisse <jglisse@redhat.com> - 2016-03-08 21:00 +0100
  [PATCH v12 12/29] HMM: add dirty range helper (toggle dirty bit inside mirror page table) v2. Jérôme Glisse <jglisse@redhat.com> - 2016-03-08 21:00 +0100
  [PATCH v12 20/29] HMM: handle HMM device page table entry on mirror page table fault and update. Jérôme Glisse <jglisse@redhat.com> - 2016-03-08 21:00 +0100

csiph-web