Re: [HMM v15 13/16] mm/hmm/migrate: new memory migration helper for use with device memory v2

Path	csiph.com!eternal-september.org!feeder.eternal-september.org!aioe.org!news.servidellagleba.it!bofh.it!news.nic.it!robomod
From	David Nellans <dnellans@nvidia.com>
Newsgroups	linux.kernel
Subject	Re: [HMM v15 13/16] mm/hmm/migrate: new memory migration helper for use with device memory v2
Date	Fri, 06 Jan 2017 17:50:02 +0100
Message-ID	<sWFSi-4A4-23@gated-at.bofh.it> (permalink)
References	<sWEWd-3VU-3@gated-at.bofh.it> <sWEWe-3VU-9@gated-at.bofh.it>
X-Original-To	Jérôme Glisse <jglisse@redhat.com>, <akpm@linux-foundation.org>, <linux-kernel@vger.kernel.org>, <linux-mm@kvack.org>
X-PGP-Universal	processed; by hqnvupgp07.nvidia.com on Thu, 05 Jan 2017 20:42:27 -0800
User-Agent	Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1
MIME-Version	1.0
X-Originating-IP	[10.20.174.107]
X-Clientproxiedby	HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL103.nvidia.com (172.20.187.11)
Content-Type	text/plain; charset="utf-8"
Content-Transfer-Encoding	8BIT
Sender	robomod@news.nic.it
List-ID	<linux-kernel.vger.kernel.org>
X-Mailing-List	linux-kernel@vger.kernel.org
Approved	robomod@news.nic.it
Lines	37
Organization	linux.* mail to news gateway
X-Original-Cc	John Hubbard <jhubbard@nvidia.com>, Evgeny Baskakov <ebaskakov@nvidia.com>, Mark Hairgrove <mhairgrove@nvidia.com>, Sherry Cheung <SCheung@nvidia.com>, Subhash Gutti <sgutti@nvidia.com>, Cameron Buschardt <cabuschardt@nvidia.com>, Zi Yan <zi.yan@cs.rutgers.edu>, Anshuman Khandual <khandual@linux.vnet.ibm.com>
X-Original-Date	Fri, 6 Jan 2017 10:46:09 -0600
X-Original-Message-ID	<d5c4a464-1f17-8517-3646-33dd5bf06ef5@nvidia.com>
X-Original-References	<1483721203-1678-1-git-send-email-jglisse@redhat.com> <1483721203-1678-14-git-send-email-jglisse@redhat.com>
X-Original-Sender	linux-kernel-owner@vger.kernel.org
Xref	csiph.com linux.kernel:1552953

Show key headers only | View raw


On 01/06/2017 10:46 AM, Jérôme Glisse wrote:
> This patch add a new memory migration helpers, which migrate memory
> backing a range of virtual address of a process to different memory
> (which can be allocated through special allocator). It differs from
> numa migration by working on a range of virtual address and thus by
> doing migration in chunk that can be large enough to use DMA engine
> or special copy offloading engine.
>
> Expected users are any one with heterogeneous memory where different
> memory have different characteristics (latency, bandwidth, ...). As
> an example IBM platform with CAPI bus can make use of this feature
> to migrate between regular memory and CAPI device memory. New CPU
> architecture with a pool of high performance memory not manage as
> cache but presented as regular memory (while being faster and with
> lower latency than DDR) will also be prime user of this patch.
Why should the normal page migration path (where neither src nor dest are
device private), use the hmm_migrate functionality?  11-14 are
replicating a lot of the
normal migration functionality but with special casing for HMM
requirements.  When migrating
THP's or a list of pages (your use case above), normal NUMA migration
is going to want to do this as fast as possible too (see Zi Yan's
patches for multi-threading normal
migrations & prototype of using intel IOAT for transfers, he sees 3-5x
speedup).

If the intention is to provide a common interface hook for migration to
use DMA acceleration
(which is a good idea), it probably shouldn't be special cased inside
HMM functionality.
For example, using the intel IOAT for migration DMA has nothing to do
with HMM
whatsoever. We need a normal migration path interface to allow DMA that
isn't tied
to HMM.

Thread

[HMM v15 13/16] mm/hmm/migrate: new memory migration helper for use with device memory v2 Jérôme Glisse <jglisse@redhat.com> - 2017-01-06 16:50 +0100
  Re: [HMM v15 13/16] mm/hmm/migrate: new memory migration helper for  use with device memory v2 David Nellans <dnellans@nvidia.com> - 2017-01-06 17:50 +0100
    Re: [HMM v15 13/16] mm/hmm/migrate: new memory migration helper for  use with device memory v2 Jerome Glisse <jglisse@redhat.com> - 2017-01-06 18:20 +0100
      Re: [HMM v15 13/16] mm/hmm/migrate: new memory migration helper for  use with device memory v2 David Nellans <dnellans@nvidia.com> - 2017-01-10 16:40 +0100
        Re: [HMM v15 13/16] mm/hmm/migrate: new memory migration helper for  use with device memory v2 Jerome Glisse <jglisse@redhat.com> - 2017-01-10 18:00 +0100

csiph-web