Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1508023

Re: [RFC 3/8] mm: Isolate coherent device memory nodes from HugeTLB allocation paths

Show all headers | View raw

On 25/10/16 15:15, Aneesh Kumar K.V wrote:
> Dave Hansen <dave.hansen@intel.com> writes:
> 
>> On 10/23/2016 09:31 PM, Anshuman Khandual wrote:
>>> This change is part of the isolation requiring coherent device memory nodes
>>> implementation.
>>>
>>> Isolation seeking coherent device memory node requires allocation isolation
>>> from implicit memory allocations from user space. Towards that effect, the
>>> memory should not be used for generic HugeTLB page pool allocations. This
>>> modifies relevant functions to skip all coherent memory nodes present on
>>> the system during allocation, freeing and auditing for HugeTLB pages.
>>
>> This seems really fragile.  You had to hit, what, 18 call sites?  What
>> are the odds that this is going to stay working?
> 
> 
> I guess a better approach is to introduce new node_states entry such
> that we have one that excludes coherent device memory numa nodes. One
> possibility is to add N_SYSTEM_MEMORY and N_MEMORY.
> 
> Current N_MEMORY becomes N_SYSTEM_MEMORY and N_MEMORY includes
> system and device/any other memory which is coherent.
> 

I thought of this as well, but I would rather see N_COHERENT_MEMORY
as a flag. The idea being that some device memory is a part of
N_MEMORY, but N_COHERENT_MEMORY gives it additional attributes

> All the isolation can then be achieved based on the nodemask_t used for
> allocation. So for allocations we want to avoid from coherent device we
> use N_SYSTEM_MEMORY mask or a derivative of that and where we are ok to
> allocate from CDM with fallbacks we use N_MEMORY.
> 

I suspect its going to be easier to exclude N_COHERENT_MEMORY.

> All nodes zonelist will have zones from the coherent device nodes but we
> will not end up allocating from coherent device node zone due to the
> node mask used.
> 
> 
> This will also make sure we end up allocating from the correct coherent
> device numa node in the presence of multiple of them based on the
> distance of the coherent device node from the current executing numa
> node.
> 

The idea is good overall, but I think its going to be good to document
the exclusions with the flags

Balbir Singh.

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[RFC 0/8] Define coherent device memory node Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2016-10-24 06:40 +0200
  [RFC 3/8] mm: Isolate coherent device memory nodes from HugeTLB allocation paths Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2016-10-24 06:40 +0200
    Re: [RFC 3/8] mm: Isolate coherent device memory nodes from HugeTLB  allocation paths Dave Hansen <dave.hansen@intel.com> - 2016-10-24 19:20 +0200
      Re: [RFC 3/8] mm: Isolate coherent device memory nodes from HugeTLB allocation paths "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com> - 2016-10-25 06:20 +0200
        Re: [RFC 3/8] mm: Isolate coherent device memory nodes from HugeTLB  allocation paths Balbir Singh <bsingharora@gmail.com> - 2016-10-25 09:20 +0200
          Re: [RFC 3/8] mm: Isolate coherent device memory nodes from HugeTLB  allocation paths Balbir Singh <bsingharora@gmail.com> - 2016-10-25 09:30 +0200
  [RFC 7/8] mm: Add a new migration function migrate_virtual_range() Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2016-10-24 06:40 +0200
  [DEBUG 06/10] mm: Export definition of 'zone_names' array through mmzone.h Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2016-10-24 06:50 +0200
  [DEBUG 00/10] Test and debug patches for coherent device memory Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2016-10-24 06:50 +0200
    [DEBUG 02/10] powerpc/mm: Create numa nodes for hotplug memory Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2016-10-24 06:50 +0200
    [DEBUG 03/10] powerpc/mm: Allow memory hotplug into a memory less node Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2016-10-24 06:50 +0200
    [DEBUG 08/10] powerpc: Enable CONFIG_MOVABLE_NODE for PPC64 platform Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2016-10-24 06:50 +0200
    [DEBUG 01/10] dt-bindings: Add doc for ibm,hotplug-aperture Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2016-10-24 06:50 +0200
    [DEBUG 05/10] powerpc/mm: Identify isolation seeking coherent memory nodes during boot Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2016-10-24 06:50 +0200
    [DEBUG 10/10] test: Add a script to perform random VMA migrations across nodes Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2016-10-24 06:50 +0200
    [DEBUG 07/10] mm: Add debugfs interface to dump each node's zonelist information Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2016-10-24 06:50 +0200
    [DEBUG 04/10] mm: Enable CONFIG_MOVABLE_NODE on powerpc Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2016-10-24 06:50 +0200
    [DEBUG 09/10] drivers: Add two drivers for coherent device memory tests Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2016-10-24 06:50 +0200
  Re: [RFC 0/8] Define coherent device memory node Jerome Glisse <j.glisse@gmail.com> - 2016-10-24 19:10 +0200
    Re: [RFC 0/8] Define coherent device memory node "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com> - 2016-10-25 06:30 +0200
      Re: [RFC 0/8] Define coherent device memory node Jerome Glisse <j.glisse@gmail.com> - 2016-10-25 17:20 +0200
        Re: [RFC 0/8] Define coherent device memory node "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com> - 2016-10-26 13:10 +0200
          Re: [RFC 0/8] Define coherent device memory node Jerome Glisse <j.glisse@gmail.com> - 2016-10-26 18:10 +0200
    Re: [RFC 0/8] Define coherent device memory node "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com> - 2016-10-25 07:10 +0200
      Re: [RFC 0/8] Define coherent device memory node Jerome Glisse <j.glisse@gmail.com> - 2016-10-25 17:40 +0200
        Re: [RFC 0/8] Define coherent device memory node "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com> - 2016-10-25 19:40 +0200
          Re: [RFC 0/8] Define coherent device memory node Jerome Glisse <j.glisse@gmail.com> - 2016-10-25 21:00 +0200
            Re: [RFC 0/8] Define coherent device memory node Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2016-10-26 13:20 +0200
              Re: [RFC 0/8] Define coherent device memory node Jerome Glisse <j.glisse@gmail.com> - 2016-10-26 18:10 +0200
            Re: [RFC 0/8] Define coherent device memory node Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2016-10-26 15:00 +0200
              Re: [RFC 0/8] Define coherent device memory node Jerome Glisse <j.glisse@gmail.com> - 2016-10-26 18:30 +0200
    Re: [RFC 0/8] Define coherent device memory node Balbir Singh <bsingharora@gmail.com> - 2016-10-25 14:10 +0200
      Re: [RFC 0/8] Define coherent device memory node Jerome Glisse <j.glisse@gmail.com> - 2016-10-25 17:30 +0200
  Re: [RFC 0/8] Define coherent device memory node Dave Hansen <dave.hansen@intel.com> - 2016-10-24 20:10 +0200
    Re: [RFC 0/8] Define coherent device memory node David Nellans <dnellans@nvidia.com> - 2016-10-24 20:40 +0200
      Re: [RFC 0/8] Define coherent device memory node Dave Hansen <dave.hansen@intel.com> - 2016-10-24 21:40 +0200

csiph-web