Re: [HMM v13 00/18] HMM (Heterogeneous Memory Management) v13

From	"Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
Newsgroups	linux.kernel
Subject	Re: [HMM v13 00/18] HMM (Heterogeneous Memory Management) v13
Date	2016-11-19 16:00 +0100
Message-ID	<sFfhv-6fj-5@gated-at.bofh.it> (permalink)
References	<sEUZr-1B7-3@gated-at.bofh.it> <sF20V-65c-9@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

John Hubbard <jhubbard@nvidia.com> writes:

> On Fri, 18 Nov 2016, Jérôme Glisse wrote:
>
>> Cliff note: HMM offers 2 things (each standing on its own). First
>> it allows to use device memory transparently inside any process
>> without any modifications to process program code. Second it allows
>> to mirror process address space on a device.
>> 
>> Change since v12 is the use of struct page for device memory even if
>> the device memory is not accessible by the CPU (because of limitation
>> impose by the bus between the CPU and the device).
>> 
>> Using struct page means that their are minimal changes to core mm
>> code. HMM build on top of ZONE_DEVICE to provide struct page, it
>> adds new features to ZONE_DEVICE. The first 7 patches implement
>> those changes.
>> 
>> Rest of patchset is divided into 3 features that can each be use
>> independently from one another. First is the process address space
>> mirroring (patch 9 to 13), this allow to snapshot CPU page table
>> and to keep the device page table synchronize with the CPU one.
>> 
>> Second is a new memory migration helper which allow migration of
>> a range of virtual address of a process. This memory migration
>> also allow device to use their own DMA engine to perform the copy
>> between the source memory and destination memory. This can be
>> usefull even outside HMM context in many usecase.
>> 
>> Third part of the patchset (patch 17-18) is a set of helper to
>> register a ZONE_DEVICE node and manage it. It is meant as a
>> convenient helper so that device drivers do not each have to
>> reimplement over and over the same boiler plate code.
>> 
>> 
>> I am hoping that this can now be consider for inclusion upstream.
>> Bottom line is that without HMM we can not support some of the new
>> hardware features on x86 PCIE. I do believe we need some solution
>> to support those features or we won't be able to use such hardware
>> in standard like C++17, OpenCL 3.0 and others.
>> 
>> I have been working with NVidia to bring up this feature on their
>> Pascal GPU. There are real hardware that you can buy today that
>> could benefit from HMM. We also intend to leverage this inside the
>> open source nouveau driver.
>> 
>
> Hi,
>
> We (NVIDIA engineering) have been working closely with Jerome on this for 
> several years now, and I wanted to mention that NVIDIA is committed to 
> using HMM. We've done initial testing of this patchset on Pascal GPUs (a 
> bit more detail below) and it is looking good.
>   

This can also be used on IBM platforms like Minsky (
http://www.tomshardware.com/news/ibm-power8-nvidia-tesla-p100-minsky,32661.html
)

There is also discussion around using this for device accelerated page
migration. That can help with coherent device memory node work.
(https://lkml.kernel.org/r/1477283517-2504-1-git-send-email-khandual@linux.vnet.ibm.com)

-aneesh

Back to linux.kernel | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

[HMM v13 00/18] HMM (Heterogeneous Memory Management) v13 Jérôme Glisse <jglisse@redhat.com> - 2016-11-18 18:20 +0100
  [HMM v13 13/18] mm/hmm/mirror: device page fault handler Jérôme Glisse <jglisse@redhat.com> - 2016-11-18 18:20 +0100
  [HMM v13 12/18] mm/hmm/mirror: helper to snapshot CPU page table Jérôme Glisse <jglisse@redhat.com> - 2016-11-18 18:30 +0100
  [HMM v13 02/18] mm/ZONE_DEVICE/unaddressable: add support for un-addressable device memory Jérôme Glisse <jglisse@redhat.com> - 2016-11-18 18:30 +0100
    Re: [HMM v13 02/18] mm/ZONE_DEVICE/unaddressable: add support for  un-addressable device memory Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2016-11-21 09:10 +0100
      Re: [HMM v13 02/18] mm/ZONE_DEVICE/unaddressable: add support for  un-addressable device memory Jerome Glisse <jglisse@redhat.com> - 2016-11-21 13:40 +0100
        Re: [HMM v13 02/18] mm/ZONE_DEVICE/unaddressable: add support for  un-addressable device memory Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2016-11-22 06:20 +0100
  [HMM v13 09/18] mm/hmm/mirror: mirror process address space on device with HMM helpers Jérôme Glisse <jglisse@redhat.com> - 2016-11-18 18:30 +0100
    Re: [HMM v13 09/18] mm/hmm/mirror: mirror process address space on  device with HMM helpers Balbir Singh <bsingharora@gmail.com> - 2016-11-21 03:50 +0100
      Re: [HMM v13 09/18] mm/hmm/mirror: mirror process address space on  device with HMM helpers Jerome Glisse <jglisse@redhat.com> - 2016-11-21 06:20 +0100
  [HMM v13 05/18] mm/ZONE_DEVICE/devmem_pages_remove: allow early removal of device memory Jérôme Glisse <jglisse@redhat.com> - 2016-11-18 18:30 +0100
    Re: [HMM v13 05/18] mm/ZONE_DEVICE/devmem_pages_remove: allow early  removal of device memory Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2016-11-21 11:40 +0100
      Re: [HMM v13 05/18] mm/ZONE_DEVICE/devmem_pages_remove: allow early  removal of device memory Jerome Glisse <jglisse@redhat.com> - 2016-11-21 13:40 +0100
        Re: [HMM v13 05/18] mm/ZONE_DEVICE/devmem_pages_remove: allow early  removal of device memory Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2016-11-22 06:00 +0100
  [HMM v13 04/18] mm/ZONE_DEVICE/free-page: callback when page is freed Jérôme Glisse <jglisse@redhat.com> - 2016-11-18 18:30 +0100
    Re: [HMM v13 04/18] mm/ZONE_DEVICE/free-page: callback when page is  freed Balbir Singh <bsingharora@gmail.com> - 2016-11-21 03:00 +0100
      Re: [HMM v13 04/18] mm/ZONE_DEVICE/free-page: callback when page is  freed Jerome Glisse <jglisse@redhat.com> - 2016-11-21 06:00 +0100
    Re: [HMM v13 04/18] mm/ZONE_DEVICE/free-page: callback when page is  freed Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2016-11-21 09:30 +0100
      Re: [HMM v13 04/18] mm/ZONE_DEVICE/free-page: callback when page is  freed Jerome Glisse <jglisse@redhat.com> - 2016-11-21 13:40 +0100
        Re: [HMM v13 04/18] mm/ZONE_DEVICE/free-page: callback when page is  freed Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2016-11-22 06:10 +0100
  [HMM v13 10/18] mm/hmm/mirror: add range lock helper, prevent CPU page table update for the range Jérôme Glisse <jglisse@redhat.com> - 2016-11-18 18:30 +0100
  [HMM v13 06/18] mm/ZONE_DEVICE/unaddressable: add special swap for unaddressable Jérôme Glisse <jglisse@redhat.com> - 2016-11-18 18:30 +0100
    Re: [HMM v13 06/18] mm/ZONE_DEVICE/unaddressable: add special swap  for unaddressable Balbir Singh <bsingharora@gmail.com> - 2016-11-21 03:10 +0100
      Re: [HMM v13 06/18] mm/ZONE_DEVICE/unaddressable: add special swap  for unaddressable Jerome Glisse <jglisse@redhat.com> - 2016-11-21 06:10 +0100
        Re: [HMM v13 06/18] mm/ZONE_DEVICE/unaddressable: add special swap  for unaddressable Balbir Singh <bsingharora@gmail.com> - 2016-11-22 03:30 +0100
          Re: [HMM v13 06/18] mm/ZONE_DEVICE/unaddressable: add special swap  for unaddressable Jerome Glisse <jglisse@redhat.com> - 2016-11-22 15:00 +0100
      Re: [HMM v13 06/18] mm/ZONE_DEVICE/unaddressable: add special swap  for unaddressable Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2016-11-21 12:20 +0100
    Re: [HMM v13 06/18] mm/ZONE_DEVICE/unaddressable: add special swap  for unaddressable Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2016-11-21 12:00 +0100
      Re: [HMM v13 06/18] mm/ZONE_DEVICE/unaddressable: add special swap  for unaddressable Jerome Glisse <jglisse@redhat.com> - 2016-11-21 13:50 +0100
        Re: [HMM v13 06/18] mm/ZONE_DEVICE/unaddressable: add special swap  for unaddressable Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2016-11-22 05:50 +0100
          Re: [HMM v13 06/18] mm/ZONE_DEVICE/unaddressable: add special swap  for unaddressable Jerome Glisse <jglisse@redhat.com> - 2016-11-24 15:00 +0100
  [HMM v13 11/18] mm/hmm/mirror: add range monitor helper, to monitor CPU page table update Jérôme Glisse <jglisse@redhat.com> - 2016-11-18 18:30 +0100
  Re: [HMM v13 00/18] HMM (Heterogeneous Memory Management) v13 John Hubbard <jhubbard@nvidia.com> - 2016-11-19 01:50 +0100
    Re: [HMM v13 00/18] HMM (Heterogeneous Memory Management) v13 "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com> - 2016-11-19 16:00 +0100

csiph-web