Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1478824 > unrolled thread

[PATCH v2 00/33] Enable Intel Resource Allocation in Resource Director Technology

Back to article view | Back to linux.kernel

Contents

  [PATCH v2 00/33] Enable Intel Resource Allocation in Resource Director Technology  "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:00 +0200
    [PATCH v2 32/33] MAINTAINERS: Add maintainer for Intel RDT resource allocation "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:00 +0200
    [PATCH v2 30/33] x86/intel_rdt_rdtgroup.c: Process schemata input from resctrl interface "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:00 +0200
      Re: [PATCH v2 30/33] x86/intel_rdt_rdtgroup.c: Process schemata  input from resctrl interface Thomas Gleixner <tglx@linutronix.de> - 2016-09-09 00:30 +0200
    [PATCH v2 31/33] Documentation/kernel-parameters: Add kernel parameter "resctrl" for CAT "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:00 +0200
      Re: [PATCH v2 31/33] Documentation/kernel-parameters: Add kernel  parameter "resctrl" for CAT Thomas Gleixner <tglx@linutronix.de> - 2016-09-09 00:30 +0200
    [PATCH v2 28/33] x86/intel_rdt_rdtgroup.c: Read and write cpus "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:10 +0200
      Re: [PATCH v2 28/33] x86/intel_rdt_rdtgroup.c: Read and write cpus Thomas Gleixner <tglx@linutronix.de> - 2016-09-08 22:30 +0200
    [PATCH v2 24/33] x86/intel_rdt_rdtgroup.c: Create info directory "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:10 +0200
      Re: [PATCH v2 24/33] x86/intel_rdt_rdtgroup.c: Create info  directory Thomas Gleixner <tglx@linutronix.de> - 2016-09-08 18:10 +0200
    [PATCH v2 11/33] x86/intel_rdt: Hot cpu support for Cache Allocation "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:10 +0200
      Re: [PATCH v2 11/33] x86/intel_rdt: Hot cpu support for Cache  Allocation Thomas Gleixner <tglx@linutronix.de> - 2016-09-08 12:10 +0200
      Re: [PATCH v2 11/33] x86/intel_rdt: Hot cpu support for Cache Allocation Nilay Vaish <nilayvaish@gmail.com> - 2016-09-13 20:20 +0200
        RE: [PATCH v2 11/33] x86/intel_rdt: Hot cpu support for Cache  Allocation "Luck, Tony" <tony.luck@intel.com> - 2016-09-13 21:20 +0200
    [PATCH v2 13/33] Define CONFIG_INTEL_RDT "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:10 +0200
      Re: [PATCH v2 13/33] Define CONFIG_INTEL_RDT Thomas Gleixner <tglx@linutronix.de> - 2016-09-08 12:20 +0200
    [PATCH v2 08/33] x86/intel_rdt: Add Class of service management "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:10 +0200
      Re: [PATCH v2 08/33] x86/intel_rdt: Add Class of service  management Thomas Gleixner <tglx@linutronix.de> - 2016-09-08 11:00 +0200
      Re: [PATCH v2 08/33] x86/intel_rdt: Add Class of service management Nilay Vaish <nilayvaish@gmail.com> - 2016-09-12 18:10 +0200
    [PATCH v2 22/33] x86/intel_rdt.c: Extend RDT to per cache and per resources "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:10 +0200
      Re: [PATCH v2 22/33] x86/intel_rdt.c: Extend RDT to per cache and  per resources Thomas Gleixner <tglx@linutronix.de> - 2016-09-08 17:10 +0200
      Re: [PATCH v2 22/33] x86/intel_rdt.c: Extend RDT to per cache and per  resources Dave Hansen <dave.hansen@intel.com> - 2016-09-14 01:00 +0200
    [PATCH v2 20/33] x86/intel_rdt.h: Header for inter_rdt.c "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:10 +0200
      Re: [PATCH v2 20/33] x86/intel_rdt.h: Header for inter_rdt.c Thomas Gleixner <tglx@linutronix.de> - 2016-09-08 14:40 +0200
    [PATCH v2 10/33] x86/intel_rdt: Implement scheduling support for Intel RDT "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:10 +0200
      Re: [PATCH v2 10/33] x86/intel_rdt: Implement scheduling support  for Intel RDT Thomas Gleixner <tglx@linutronix.de> - 2016-09-08 12:00 +0200
        RE: [PATCH v2 10/33] x86/intel_rdt: Implement scheduling support  for Intel RDT "Yu, Fenghua" <fenghua.yu@intel.com> - 2016-09-15 23:40 +0200
          RE: [PATCH v2 10/33] x86/intel_rdt: Implement scheduling support  for Intel RDT "Luck, Tony" <tony.luck@intel.com> - 2016-09-15 23:50 +0200
      Re: [PATCH v2 10/33] x86/intel_rdt: Implement scheduling support for  Intel RDT Nilay Vaish <nilayvaish@gmail.com> - 2016-09-13 20:00 +0200
    [PATCH v2 16/33] x86/intel_rdt: Class of service and capacity bitmask management for CDP "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:10 +0200
      Re: [PATCH v2 16/33] x86/intel_rdt: Class of service and capacity  bitmask management for CDP Thomas Gleixner <tglx@linutronix.de> - 2016-09-08 12:40 +0200
    [PATCH v2 04/33] drivers/base/cacheinfo.c: Export some cacheinfo functions for others to use "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:10 +0200
      Re: [PATCH v2 04/33] drivers/base/cacheinfo.c: Export some cacheinfo  functions for others to use Thomas Gleixner <tglx@linutronix.de> - 2016-09-08 10:30 +0200
    [PATCH v2 25/33] include/linux/resctrl.h: Define fork and exit functions in a new header file "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:10 +0200
      Re: [PATCH v2 25/33] include/linux/resctrl.h: Define fork and exit  functions in a new header file Thomas Gleixner <tglx@linutronix.de> - 2016-09-08 18:20 +0200
    [PATCH v2 27/33] x86/intel_rdt_rdtgroup.c: Implement resctrl file system commands "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:10 +0200
      Re: [PATCH v2 27/33] x86/intel_rdt_rdtgroup.c: Implement resctrl  file system commands Thomas Gleixner <tglx@linutronix.de> - 2016-09-08 22:20 +0200
      Re: [PATCH v2 27/33] x86/intel_rdt_rdtgroup.c: Implement resctrl  file system commands Fenghua Yu <fenghua.yu@intel.com> - 2016-09-09 00:30 +0200
    [PATCH v2 29/33] x86/intel_rdt_rdtgroup.c: Tasks iterator and write "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:10 +0200
      Re: [PATCH v2 29/33] x86/intel_rdt_rdtgroup.c: Tasks iterator and  write Thomas Gleixner <tglx@linutronix.de> - 2016-09-08 23:00 +0200
    [PATCH v2 17/33] x86/intel_rdt: Hot cpu update for code data prioritization "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:10 +0200
      Re: [PATCH v2 17/33] x86/intel_rdt: Hot cpu update for code data  prioritization Thomas Gleixner <tglx@linutronix.de> - 2016-09-08 12:40 +0200
    [PATCH v2 06/33] Documentation, x86: Documentation for Intel resource allocation user interface "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:10 +0200
      Re: [PATCH v2 06/33] Documentation, x86: Documentation for Intel  resource allocation user interface Borislav Petkov <bp@suse.de> - 2016-09-08 13:30 +0200
      Re: [PATCH v2 06/33] Documentation, x86: Documentation for Intel  resource allocation user interface Fenghua Yu <fenghua.yu@intel.com> - 2016-09-09 00:20 +0200
        Re: [PATCH v2 06/33] Documentation, x86: Documentation for Intel  resource allocation user interface Fenghua Yu <fenghua.yu@intel.com> - 2016-09-09 06:30 +0200
          RE: [PATCH v2 06/33] Documentation, x86: Documentation for Intel  resource allocation user interface "Luck, Tony" <tony.luck@intel.com> - 2016-09-09 20:10 +0200
            RE: [PATCH v2 06/33] Documentation, x86: Documentation for Intel  resource allocation user interface "Luck, Tony" <tony.luck@intel.com> - 2016-09-10 00:10 +0200
              RE: [PATCH v2 06/33] Documentation, x86: Documentation for Intel  resource allocation user interface "Yu, Fenghua" <fenghua.yu@intel.com> - 2016-09-10 02:40 +0200
                Re: [PATCH v2 06/33] Documentation, x86: Documentation for Intel  resource allocation user interface Thomas Gleixner <tglx@linutronix.de> - 2016-09-13 15:40 +0200
    [PATCH v2 03/33] x86, intel_cacheinfo: Enable cache id in x86 "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:10 +0200
    [PATCH v2 09/33] x86/intel_rdt: Add L3 cache capacity bitmask management "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:10 +0200
      Re: [PATCH v2 09/33] x86/intel_rdt: Add L3 cache capacity bitmask  management Thomas Gleixner <tglx@linutronix.de> - 2016-09-08 11:50 +0200
      Re: [PATCH v2 09/33] x86/intel_rdt: Add L3 cache capacity bitmask management Nilay Vaish <nilayvaish@gmail.com> - 2016-09-12 18:20 +0200
    [PATCH v2 07/33] x86/intel_rdt: Add support for Cache Allocation detection "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:10 +0200
      Re: [PATCH v2 07/33] x86/intel_rdt: Add support for Cache Allocation  detection Borislav Petkov <bp@suse.de> - 2016-09-08 14:00 +0200
        RE: [PATCH v2 07/33] x86/intel_rdt: Add support for Cache  Allocation detection "Yu, Fenghua" <fenghua.yu@intel.com> - 2016-09-08 19:00 +0200
          Re: [PATCH v2 07/33] x86/intel_rdt: Add support for Cache Allocation  detection Borislav Petkov <bp@suse.de> - 2016-09-08 19:20 +0200
      Re: [PATCH v2 07/33] x86/intel_rdt: Add support for Cache Allocation  detection Thomas Gleixner <tglx@linutronix.de> - 2016-09-08 15:20 +0200
        RE: [PATCH v2 07/33] x86/intel_rdt: Add support for Cache  Allocation detection "Yu, Fenghua" <fenghua.yu@intel.com> - 2016-09-08 16:00 +0200
      Re: [PATCH v2 07/33] x86/intel_rdt: Add support for Cache Allocation  detection Dave Hansen <dave.hansen@intel.com> - 2016-09-14 00:50 +0200
        Re: [PATCH v2 07/33] x86/intel_rdt: Add support for Cache Allocation  detection "Luck, Tony" <tony.luck@intel.com> - 2016-09-14 01:00 +0200
          Re: [PATCH v2 07/33] x86/intel_rdt: Add support for Cache Allocation  detection Dave Hansen <dave.hansen@intel.com> - 2016-09-14 01:10 +0200
    [PATCH v2 01/33] cacheinfo: Introduce cache id "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:10 +0200
      Re: [PATCH v2 01/33] cacheinfo: Introduce cache id Nilay Vaish <nilayvaish@gmail.com> - 2016-09-09 17:10 +0200
    [PATCH v2 05/33] x86/intel_rdt: Cache Allocation documentation "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:10 +0200
    [PATCH v2 19/33] magic number for resctrl file system "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:10 +0200
      Re: [PATCH v2 19/33] magic number for resctrl file system Thomas Gleixner <tglx@linutronix.de> - 2016-09-08 12:50 +0200
        Re: [PATCH v2 19/33] magic number for resctrl file system Borislav Petkov <bp@alien8.de> - 2016-09-08 12:50 +0200
    [PATCH v2 21/33] x86/intel_rdt_rdtgroup.h: Header for user interface "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:10 +0200
      Re: [PATCH v2 21/33] x86/intel_rdt_rdtgroup.h: Header for user  interface Thomas Gleixner <tglx@linutronix.de> - 2016-09-08 14:50 +0200
    [PATCH v2 26/33] Task fork and exit for rdtgroup "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:10 +0200
      Re: [PATCH v2 26/33] Task fork and exit for rdtgroup Thomas Gleixner <tglx@linutronix.de> - 2016-09-08 21:50 +0200
      Re: [PATCH v2 26/33] Task fork and exit for rdtgroup Dave Hansen <dave.hansen@intel.com> - 2016-09-14 01:20 +0200
        Re: [PATCH v2 26/33] Task fork and exit for rdtgroup "Luck, Tony" <tony.luck@intel.com> - 2016-09-14 01:40 +0200
          Re: [PATCH v2 26/33] Task fork and exit for rdtgroup Dave Hansen <dave.hansen@intel.com> - 2016-09-14 16:30 +0200
    [PATCH v2 18/33] sched.h: Add rg_list and rdtgroup in task_struct "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:10 +0200
      Re: [PATCH v2 18/33] sched.h: Add rg_list and rdtgroup in  task_struct Thomas Gleixner <tglx@linutronix.de> - 2016-09-08 12:40 +0200
    [PATCH v2 02/33] Documentation, ABI: Add a document entry for cache id "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:10 +0200
      Re: [PATCH v2 02/33] Documentation, ABI: Add a document entry for  cache id Thomas Gleixner <tglx@linutronix.de> - 2016-09-08 21:40 +0200
        Re: [PATCH v2 02/33] Documentation, ABI: Add a document entry for  cache id Nilay Vaish <nilayvaish@gmail.com> - 2016-09-09 17:20 +0200
    [PATCH v2 12/33] x86/intel_rdt: Intel haswell Cache Allocation enumeration "Fenghua Yu" <fenghua.yu@intel.com> - 2016-09-08 09:10 +0200
      Re: [PATCH v2 12/33] x86/intel_rdt: Intel haswell Cache Allocation  enumeration Thomas Gleixner <tglx@linutronix.de> - 2016-09-08 12:20 +0200

Page 3 of 5 — ← Prev page 1 2 [3] 4 5  Next page →

#1478842 — [PATCH v2 17/33] x86/intel_rdt: Hot cpu update for code data prioritization

From: Vikas Shivappa <vikas.shivappa@linux.intel.com>

Updates hot cpu notification handling for code data prioritization(cdp).
The capacity bitmask(cbm) is global for both data and instruction and we
need to update the new online package with all the cbms by writing to
the IA32_L3_QOS_n MSRs.

Signed-off-by: Vikas Shivappa <vikas.shivappa@linux.intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Reviewed-by: Tony Luck <tony.luck@intel.com>
---
 arch/x86/kernel/cpu/intel_rdt.c | 27 +++++++++++++++++++++------
 1 file changed, 21 insertions(+), 6 deletions(-)

diff --git a/arch/x86/kernel/cpu/intel_rdt.c b/arch/x86/kernel/cpu/intel_rdt.c
index 9cee3fe..1bcff29 100644
--- a/arch/x86/kernel/cpu/intel_rdt.c
+++ b/arch/x86/kernel/cpu/intel_rdt.c
@@ -223,6 +223,26 @@ static inline bool rdt_cpumask_update(int cpu)
 	return false;
 }
 
+static void cbm_update_msr(u32 index)
+{
+	struct rdt_remote_data info;
+	int dindex;
+
+	dindex = DCBM_TABLE_INDEX(index);
+	if (cctable[dindex].clos_refcnt) {
+
+		info.msr = CBM_FROM_INDEX(dindex);
+		info.val = cctable[dindex].cbm;
+		msr_cpu_update((void *) &info);
+
+		if (cdp_enabled) {
+			info.msr = __ICBM_MSR_INDEX(index);
+			info.val = cctable[dindex + 1].cbm;
+			msr_cpu_update((void *) &info);
+		}
+	}
+}
+
 /*
  * cbm_update_msrs() - Updates all the existing IA32_L3_MASK_n MSRs
  * which are one per CLOSid on the current package.
@@ -230,15 +250,10 @@ static inline bool rdt_cpumask_update(int cpu)
 static void cbm_update_msrs(void *dummy)
 {
 	int maxid = cconfig.max_closid;
-	struct rdt_remote_data info;
 	unsigned int i;
 
 	for (i = 0; i < maxid; i++) {
-		if (cctable[i].clos_refcnt) {
-			info.msr = CBM_FROM_INDEX(i);
-			info.val = cctable[i].cbm;
-			msr_cpu_update((void *) &info);
-		}
+		cbm_update_msr(i);
 	}
 }
 
-- 
2.5.0

[toc] | [prev] | [next] | [standalone]

#1479061 — Re: [PATCH v2 17/33] x86/intel_rdt: Hot cpu update for code data prioritization

On Thu, 8 Sep 2016, Fenghua Yu wrote:
> Updates hot cpu notification handling for code data prioritization(cdp).

Some more useless information

> The capacity bitmask(cbm) is global for both data and instruction and we
> need to update the new online package with all the cbms by writing to
> the IA32_L3_QOS_n MSRs.

If I wouldn't know the details of the hardware then this explanation would
make me run away screaming ....

> +static void cbm_update_msr(u32 index)
> +{
> +	struct rdt_remote_data info;
> +	int dindex;
> +
> +	dindex = DCBM_TABLE_INDEX(index);
> +	if (cctable[dindex].clos_refcnt) {
> +
> +		info.msr = CBM_FROM_INDEX(dindex);
> +		info.val = cctable[dindex].cbm;
> +		msr_cpu_update((void *) &info);
> +
> +		if (cdp_enabled) {
> +			info.msr = __ICBM_MSR_INDEX(index);
> +			info.val = cctable[dindex + 1].cbm;
> +			msr_cpu_update((void *) &info);
> +		}
> +	}

As usual there is a complete lack of comments here.

> +}
> +
>  /*
>   * cbm_update_msrs() - Updates all the existing IA32_L3_MASK_n MSRs
>   * which are one per CLOSid on the current package.
> @@ -230,15 +250,10 @@ static inline bool rdt_cpumask_update(int cpu)
>  static void cbm_update_msrs(void *dummy)
>  {
>  	int maxid = cconfig.max_closid;
> -	struct rdt_remote_data info;
>  	unsigned int i;
>  
>  	for (i = 0; i < maxid; i++) {
> -		if (cctable[i].clos_refcnt) {
> -			info.msr = CBM_FROM_INDEX(i);
> -			info.val = cctable[i].cbm;
> -			msr_cpu_update((void *) &info);
> -		}
> +		cbm_update_msr(i);
>  	}

The curly braces can go as well.

Thanks,

	tglx

[toc] | [prev] | [next] | [standalone]

#1478844 — [PATCH v2 06/33] Documentation, x86: Documentation for Intel resource allocation user interface

From: Fenghua Yu <fenghua.yu@intel.com>

The documentation describes user interface of how to allocate resource
in Intel RDT.

Please note that the documentation covers generic user interface. Current
patch set code only implemente CAT L3. CAT L2 code will be sent later.

Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Reviewed-by: Tony Luck <tony.luck@intel.com>
---
 Documentation/x86/intel_rdt_ui.txt | 164 +++++++++++++++++++++++++++++++++++++
 1 file changed, 164 insertions(+)
 create mode 100644 Documentation/x86/intel_rdt_ui.txt

diff --git a/Documentation/x86/intel_rdt_ui.txt b/Documentation/x86/intel_rdt_ui.txt
new file mode 100644
index 0000000..27de386
--- /dev/null
+++ b/Documentation/x86/intel_rdt_ui.txt
@@ -0,0 +1,164 @@
+User Interface for Resource Allocation in Intel Resource Director Technology
+
+Copyright (C) 2016 Intel Corporation
+
+Fenghua Yu <fenghua.yu@intel.com>
+Tony Luck <tony.luck@intel.com>
+
+This feature is enabled by the CONFIG_INTEL_RDT Kconfig and the
+X86 /proc/cpuinfo flag bits "rdt", "cat_l3" and "cdp_l3".
+
+To use the feature mount the file system:
+
+ # mount -t resctrl resctrl [-o cdp,verbose] /sys/fs/resctrl
+
+mount options are:
+
+"cdp": Enable code/data prioritization in L3 cache allocations.
+
+"verbose": Output more info in the "info" file under info directory
+	and in dmesg. This is mainly for debug.
+
+
+Resource groups
+---------------
+Resource groups are represented as directories in the resctrl file
+system. The default group is the root directory. Other groups may be
+created as desired by the system administrator using the "mkdir(1)"
+command, and removed using "rmdir(1)".
+
+There are three files associated with each group:
+
+"tasks": A list of tasks that belongs to this group. Tasks can be
+	added to a group by writing the task ID to the "tasks" file
+	(which will automatically remove them from the previous
+	group to which they belonged). New tasks created by fork(2)
+	and clone(2) are added to the same group as their parent.
+	If a pid is not in any sub partition, it is in root partition
+	(i.e. default partition).
+
+"cpus": A bitmask of logical CPUs assigned to this group. Writing
+	a new mask can add/remove CPUs from this group. Added CPUs
+	are removed from their previous group. Removed ones are
+	given to the default (root) group.
+
+"schemata": A list of all the resources available to this group.
+	Each resource has its own line and format - see below for
+	details.
+
+When a task is running the following rules define which resources
+are available to it:
+
+1) If the task is a member of a non-default group, then the schemata
+for that group is used.
+
+2) Else if the task belongs to the default group, but is running on a
+CPU that is assigned to some specific group, then the schemata for
+the CPU's group is used.
+
+3) Otherwise the schemata for the default group is used.
+
+
+Schemata files - general concepts
+---------------------------------
+Each line in the file describes one resource. The line starts with
+the name of the resource, followed by specific values to be applied
+in each of the instances of that resource on the system.
+
+Cache IDs
+---------
+On current generation systems there is one L3 cache per socket and L2
+caches are generally just shared by the hyperthreads on a core, but this
+isn't an architectural requirement. We could have multiple separate L3
+caches on a socket, multiple cores could share an L2 cache. So instead
+of using "socket" or "core" to define the set of logical cpus sharing
+a resource we use a "Cache ID". At a given cache level this will be a
+unique number across the whole system (but it isn't guaranteed to be a
+contiguous sequence, there may be gaps).  To find the ID for each logical
+CPU look in /sys/devices/system/cpu/cpu*/cache/index*/id
+
+Cache Bit Masks (CBM)
+---------------------
+For cache resources we describe the portion of the cache that is available
+for allocation using a bitmask. The number of bits in the mask is defined
+by each cpu model (and may be different for different cache levels). It
+is found using CPUID, but is also provided in the "info" directory of
+the resctrl file system in "info/{resource}/max_cbm_len". X86 hardware
+requires that these masks have all the '1' bits in a contiguous block. So
+0x3, 0x6 and 0xC are legal 4-bit masks with two bits set, but 0x5, 0x9
+and 0xA are not.  On a system with a 20-bit mask each bit represents 5%
+of the capacity of the cache. You could partition the cache into four
+equal parts with masks: 0x1f, 0x3e0, 0x7c00, 0xf8000.
+
+
+L3 details (code and data prioritization disabled)
+--------------------------------------------------
+With CDP disabled the L3 schemata format is:
+
+	L3:<cache_id0>=<cbm>;<cache_id1>=<cbm>;...
+
+L3 details (CDP enabled via mount option to resctrl)
+----------------------------------------------------
+When CDP is enabled, you need to specify separate cache bit masks for
+code and data access. The generic format is:
+
+	L3:<cache_id0>=<d_cbm>,<i_cbm>;<cache_id1>=<d_cbm>,<i_cbm>;...
+
+where the d_cbm masks are for data access, and the i_cbm masks for code.
+
+
+Example 1
+---------
+On a two socket machine (one L3 cache per socket) with just four bits
+for cache bit masks
+
+# mount -t resctrl resctrl /sys/fs/resctrl
+# cd /sys/fs/resctrl
+# mkdir p0 p1
+# echo "L3:0=3;1=c" > /sys/fs/resctrl/p0/schemata
+# echo "L3:0=3;1=3" > /sys/fs/resctrl/p1/schemata
+
+The default resource group is unmodified, so we have access to all parts
+of all caches (its schemata file reads "L3:0=f;1=f").
+
+Tasks that are under the control of group "p0" may only allocate from the
+"lower" 50% on cache ID 0, and the "upper" 50% of cache ID 1.
+Tasks in group "p1" use the "lower" 50% of cache on both sockets.
+
+Example 2
+---------
+Again two sockets, but this time with a more realistic 20-bit mask.
+
+Two real time tasks pid=1234 running on processor 0 and pid=5678 running on
+processor 1 on socket 0 on a 2-socket and dual core machine. To avoid noisy
+neighbors, each of the two real-time tasks exclusively occupies one quarter
+of L3 cache on socket 0.
+
+# mount -t resctrl resctrl /sys/fs/resctrl
+# cd /sys/fs/resctrl
+
+First we reset the schemata for the default group so that the "upper"
+50% of the L3 cache on socket 0 cannot be used by ordinary tasks:
+
+# echo "L3:0=3ff;1=fffff" > schemata
+
+Next we make a resource group for our first real time task and give
+it access to the "top" 25% of the cache on socket 0.
+
+# mkdir p0
+# echo "L3:0=f8000;1=fffff" > p0/schemata
+
+Finally we move our first real time task into this resource group. We
+also use taskset(1) to ensure the task always runs on a dedicated CPU
+on socket 0. Most uses of resource groups will also constrain which
+processors tasks run on.
+
+# echo 1234 > p0/tasks
+# taskset -cp 1 1234
+
+Ditto for the second real time task (with the remaining 25% of cache):
+
+# mkdir p1
+# echo "L3:0=7c00;1=fffff" > p1/schemata
+# echo 5678 > p1/tasks
+# taskset -cp 2 5678
-- 
2.5.0

[toc] | [prev] | [next] | [standalone]

#1479115 — Re: [PATCH v2 06/33] Documentation, x86: Documentation for Intel resource allocation user interface

On Thu, Sep 08, 2016 at 02:57:00AM -0700, Fenghua Yu wrote:
> From: Fenghua Yu <fenghua.yu@intel.com>
> 
> The documentation describes user interface of how to allocate resource
> in Intel RDT.
> 
> Please note that the documentation covers generic user interface. Current
> patch set code only implemente CAT L3. CAT L2 code will be sent later.
> 
> Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
> Reviewed-by: Tony Luck <tony.luck@intel.com>
> ---
>  Documentation/x86/intel_rdt_ui.txt | 164 +++++++++++++++++++++++++++++++++++++
>  1 file changed, 164 insertions(+)
>  create mode 100644 Documentation/x86/intel_rdt_ui.txt
> 
> diff --git a/Documentation/x86/intel_rdt_ui.txt b/Documentation/x86/intel_rdt_ui.txt
> new file mode 100644
> index 0000000..27de386
> --- /dev/null
> +++ b/Documentation/x86/intel_rdt_ui.txt

Why isn't this part of Documentation/x86/intel_rdt.txt and needs to be a
separate file?

-- 
Regards/Gruss,
    Boris.

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
-- 

[toc] | [prev] | [next] | [standalone]

#1479545 — Re: [PATCH v2 06/33] Documentation, x86: Documentation for Intel resource allocation user interface

On Thu, Sep 08, 2016 at 03:01:20PM -0700, Shaohua Li wrote:
> On Thu, Sep 08, 2016 at 02:57:00AM -0700, Fenghua Yu wrote:
> > From: Fenghua Yu <fenghua.yu@intel.com>
> > 
> > The documentation describes user interface of how to allocate resource
> > in Intel RDT.
> > 
> > Please note that the documentation covers generic user interface. Current
> > patch set code only implemente CAT L3. CAT L2 code will be sent later.
> > 
> > Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
> > Reviewed-by: Tony Luck <tony.luck@intel.com>
> > ---
> >  Documentation/x86/intel_rdt_ui.txt | 164 +++++++++++++++++++++++++++++++++++++
> >  1 file changed, 164 insertions(+)
> >  create mode 100644 Documentation/x86/intel_rdt_ui.txt
> > 
> > diff --git a/Documentation/x86/intel_rdt_ui.txt b/Documentation/x86/intel_rdt_ui.txt
> > new file mode 100644
> > index 0000000..27de386
> > --- /dev/null
> > +++ b/Documentation/x86/intel_rdt_ui.txt
> > @@ -0,0 +1,164 @@
> > +User Interface for Resource Allocation in Intel Resource Director Technology
> > +
> > +Copyright (C) 2016 Intel Corporation
> > +
> > +Fenghua Yu <fenghua.yu@intel.com>
> > +Tony Luck <tony.luck@intel.com>
> > +
> > +This feature is enabled by the CONFIG_INTEL_RDT Kconfig and the
> > +X86 /proc/cpuinfo flag bits "rdt", "cat_l3" and "cdp_l3".
> > +
> > +To use the feature mount the file system:
> > +
> > + # mount -t resctrl resctrl [-o cdp,verbose] /sys/fs/resctrl
> > +
> > +mount options are:
> > +
> > +"cdp": Enable code/data prioritization in L3 cache allocations.
> > +
> > +"verbose": Output more info in the "info" file under info directory
> > +	and in dmesg. This is mainly for debug.
> > +
> > +
> > +Resource groups
> > +---------------
> > +Resource groups are represented as directories in the resctrl file
> > +system. The default group is the root directory. Other groups may be
> > +created as desired by the system administrator using the "mkdir(1)"
> > +command, and removed using "rmdir(1)".
> > +
> > +There are three files associated with each group:
> > +
> > +"tasks": A list of tasks that belongs to this group. Tasks can be
> > +	added to a group by writing the task ID to the "tasks" file
> > +	(which will automatically remove them from the previous
> > +	group to which they belonged). New tasks created by fork(2)
> > +	and clone(2) are added to the same group as their parent.
> > +	If a pid is not in any sub partition, it is in root partition
> > +	(i.e. default partition).
> Hi Fenghua,
> 
> Will you add a 'procs' interface to allow move a process into a group? Using
> the 'tasks' interface to move process is inconvenient and has race conditions
> (eg, some new threads could be escaped).

We don't plan to add a 'procs' interface for rdtgroup. We only use resctrl
interface to allocate resources.

Why the "tasks" is inconvenient? If sysadmin wants to allocte a portion of L3
for a pid, the operation in resctl is to write the pid to a "tasks". While
in 'procs', the operation is to write a partition to a pid. If considering
convenience, they are same, right?

A thread uses either default partition (in root dir) or a sub partition (in
sub-directory). Sysadmin can control that. Kernel handles race condition.
Any issue with that?

Thanks.

-Fenghua
The same 

[toc] | [prev] | [next] | [standalone]

#1479649 — Re: [PATCH v2 06/33] Documentation, x86: Documentation for Intel resource allocation user interface

On Thu, Sep 08, 2016 at 03:45:14PM -0700, Shaohua Li wrote:
> On Thu, Sep 08, 2016 at 06:17:47PM -0700, Fenghua Yu wrote:
> > On Thu, Sep 08, 2016 at 03:01:20PM -0700, Shaohua Li wrote:
> > > On Thu, Sep 08, 2016 at 02:57:00AM -0700, Fenghua Yu wrote:
> > > > From: Fenghua Yu <fenghua.yu@intel.com>
> > > > 
> > > > The documentation describes user interface of how to allocate resource
> > > > in Intel RDT.
> > > > 
> > > > Please note that the documentation covers generic user interface. Current
> > > > patch set code only implemente CAT L3. CAT L2 code will be sent later.
> > > > 
> > > > Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
> > > > Reviewed-by: Tony Luck <tony.luck@intel.com>
> > > > ---
> > > >  Documentation/x86/intel_rdt_ui.txt | 164 +++++++++++++++++++++++++++++++++++++
> > > >  1 file changed, 164 insertions(+)
> > > >  create mode 100644 Documentation/x86/intel_rdt_ui.txt
> > > > 
> > > > diff --git a/Documentation/x86/intel_rdt_ui.txt b/Documentation/x86/intel_rdt_ui.txt
> > > > new file mode 100644
> > > > index 0000000..27de386
> > > > --- /dev/null
> > > > +++ b/Documentation/x86/intel_rdt_ui.txt
> > > > @@ -0,0 +1,164 @@
> > > > +User Interface for Resource Allocation in Intel Resource Director Technology
> > > > +
> > > > +Copyright (C) 2016 Intel Corporation
> > > > +
> > > > +Fenghua Yu <fenghua.yu@intel.com>
> > > > +Tony Luck <tony.luck@intel.com>
> > > > +
> > > > +This feature is enabled by the CONFIG_INTEL_RDT Kconfig and the
> > > > +X86 /proc/cpuinfo flag bits "rdt", "cat_l3" and "cdp_l3".
> > > > +
> > > > +To use the feature mount the file system:
> > > > +
> > > > + # mount -t resctrl resctrl [-o cdp,verbose] /sys/fs/resctrl
> > > > +
> > > > +mount options are:
> > > > +
> > > > +"cdp": Enable code/data prioritization in L3 cache allocations.
> > > > +
> > > > +"verbose": Output more info in the "info" file under info directory
> > > > +	and in dmesg. This is mainly for debug.
> > > > +
> > > > +
> > > > +Resource groups
> > > > +---------------
> > > > +Resource groups are represented as directories in the resctrl file
> > > > +system. The default group is the root directory. Other groups may be
> > > > +created as desired by the system administrator using the "mkdir(1)"
> > > > +command, and removed using "rmdir(1)".
> > > > +
> > > > +There are three files associated with each group:
> > > > +
> > > > +"tasks": A list of tasks that belongs to this group. Tasks can be
> > > > +	added to a group by writing the task ID to the "tasks" file
> > > > +	(which will automatically remove them from the previous
> > > > +	group to which they belonged). New tasks created by fork(2)
> > > > +	and clone(2) are added to the same group as their parent.
> > > > +	If a pid is not in any sub partition, it is in root partition
> > > > +	(i.e. default partition).
> > > Hi Fenghua,
> > > 
> > > Will you add a 'procs' interface to allow move a process into a group? Using
> > > the 'tasks' interface to move process is inconvenient and has race conditions
> > > (eg, some new threads could be escaped).
> > 
> > We don't plan to add a 'procs' interface for rdtgroup. We only use resctrl
> > interface to allocate resources.
> > 
> > Why the "tasks" is inconvenient? If sysadmin wants to allocte a portion of L3
> > for a pid, the operation in resctl is to write the pid to a "tasks". While
> > in 'procs', the operation is to write a partition to a pid. If considering
> > convenience, they are same, right?
> > 
> > A thread uses either default partition (in root dir) or a sub partition (in
> > sub-directory). Sysadmin can control that. Kernel handles race condition.
> > Any issue with that?
> 
> I don't mean writing the 'tasks' file is inconvenient. So to move a process to
> a group, we do:
> 1. get all thread pid of the process
> 2. write every pid to 'tasks'
> 
> this is inconvenient. And if a new thread is created between 1 and 2, we don't
> put the thread to the group. Am I missing anything?

As said in this doc, "New tasks created by fork(2) and clone(2) are added
to the same group as their parent.". So the new thread created b/w 1 and 2
will automatically go to the "tasks" as the process. Later sysadming can
still move any pid to any group.

Is this convenient?

Thanks.

-Fenghua

[toc] | [prev] | [next] | [standalone]

#1480236 — RE: [PATCH v2 06/33] Documentation, x86: Documentation for Intel resource allocation user interface

> I don't think this is convenient, but it's ok. Now if we create a new thread
> between 1 and 2, the new thread is in group1. The new thread pid isn't in the
> pid list we found in 1, so after 2, the new thread still is in group 1. Truely
> sysadmin can repeat the step 1 & 2 and move the new thread to group 2, but
> there is always chance the process creates new thread between 1 and 2, and the
> new thread remains in group 1. There is no guarantee we can safely move a
> process from one group to another.

In general this is true.  But don't most threaded applications have a single thread that
is the one that spawns new threads? Typically the first thread.  Once that is moved,
any new threads will inherit the new group. So there won't be a neverending mopping
operation trying to catch up.

Even this seems outside the expected usage model for CAT where we expect the
system admin to partition the cache between resource groups at boot time and
then assign jobs (or containers, or VMs) to resource groups when they are created.

-Tony

[toc] | [prev] | [next] | [standalone]

#1480385 — RE: [PATCH v2 06/33] Documentation, x86: Documentation for Intel resource allocation user interface

> Hmm, I don't know how applications are going to use the interface. Nobody knows
> it right now. But we do have some candicate workloads which want to configure
> the cache partition at runtime, so it's not just a boot time stuff. I'm
> wondering why we have such limitation. The framework is there, it's quite easy
> to implement process move in kernel but fairly hard to get it right in
> userspace.

You are correct - if there is a need for this, it would be better done in the kernel.

I'm just not sure how to explain both a "procs" and "tasks" interface file in a way
that won't confuse people.

We have:

# echo {task-id} > tasks
  .... adds a single task to this resource group
# cat tasks
  ... shows all the tasks in this resource group

and you want:

# echo {process-id} > procs
   ... adds all threads in {process-id} to this resource group
# cat procs
  ... shows all processes (like "cat tasks" above, but only shows main thread in a multi-threads process)

[toc] | [prev] | [next] | [standalone]

#1480448 — RE: [PATCH v2 06/33] Documentation, x86: Documentation for Intel resource allocation user interface

> > Hmm, I don't know how applications are going to use the interface.
> > Nobody knows it right now. But we do have some candicate workloads
> > which want to configure the cache partition at runtime, so it's not
> > just a boot time stuff. I'm wondering why we have such limitation. The
> > framework is there, it's quite easy to implement process move in
> > kernel but fairly hard to get it right in userspace.
> 
> You are correct - if there is a need for this, it would be better done in the
> kernel.
> 
> I'm just not sure how to explain both a "procs" and "tasks" interface file in a
> way that won't confuse people.
> 
> We have:
> 
> # echo {task-id} > tasks
>   .... adds a single task to this resource group # cat tasks
>   ... shows all the tasks in this resource group
> 
> and you want:
> 
> # echo {process-id} > procs
>    ... adds all threads in {process-id} to this resource group # cat procs
>   ... shows all processes (like "cat tasks" above, but only shows main thread in
> a multi-threads process)

The advantage of "tasks" is user can allocate each thread into its own partition.
The advantage of "procs" is convenience for user to just allocate thread group
lead pid and rest of the thread group members go with the lead.

If no "procs" is really inconvenience, we may support "procs" in future.

One way to implement this is we can extend the current interface to accept
a resctrl file system mount parameter to switch b/w "procs" and "tasks" during
mount time. So the file sytem has either "procs" or "tasks" during run time. I don't think it's right to have both of them at the same time in the file system.

Is this the right way to go?

Thanks.

-Fenghua

[toc] | [prev] | [next] | [standalone]

#1482495 — Re: [PATCH v2 06/33] Documentation, x86: Documentation for Intel resource allocation user interface

On Sun, 11 Sep 2016, Shaohua Li wrote:
> On Sat, Sep 10, 2016 at 12:36:57AM +0000, Yu, Fenghua wrote:
> > One way to implement this is we can extend the current interface to accept
> > a resctrl file system mount parameter to switch b/w "procs" and "tasks" during
> > mount time. So the file sytem has either "procs" or "tasks" during run time. I don't think it's right to have both of them at the same time in the file system.
> 
> A mount option doesn't make sense, which just creates more trouble. What's
> wrong to have both of 'procs' and 'tasks' at the same time, like cgroup? I
> think it's more natural to support both. As for the content of 'procs' and
> 'tasks', we could follow how cgroup handle them.

Right. There is nothing wrong with having both, but the very first step is
to get the basic infrastructure merged. Adding 'procs' is a straight
forward add on which can be implemented on top of the primary patch
set. There is no design change required to support it later.

Thanks,

	tglx

[toc] | [prev] | [next] | [standalone]

#1478845 — [PATCH v2 03/33] x86, intel_cacheinfo: Enable cache id in x86

From: Fenghua Yu <fenghua.yu@intel.com>

Enable cache id in x86. Cache id comes from APIC ID and CPUID4.

Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Reviewed-by: Tony Luck <tony.luck@intel.com>
Acked-by: Borislav Petkov <bp@suse.com>
---
 arch/x86/kernel/cpu/intel_cacheinfo.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/arch/x86/kernel/cpu/intel_cacheinfo.c b/arch/x86/kernel/cpu/intel_cacheinfo.c
index de6626c..8dc5720 100644
--- a/arch/x86/kernel/cpu/intel_cacheinfo.c
+++ b/arch/x86/kernel/cpu/intel_cacheinfo.c
@@ -153,6 +153,7 @@ struct _cpuid4_info_regs {
 	union _cpuid4_leaf_eax eax;
 	union _cpuid4_leaf_ebx ebx;
 	union _cpuid4_leaf_ecx ecx;
+	unsigned int id;
 	unsigned long size;
 	struct amd_northbridge *nb;
 };
@@ -894,6 +895,8 @@ static void __cache_cpumap_setup(unsigned int cpu, int index,
 static void ci_leaf_init(struct cacheinfo *this_leaf,
 			 struct _cpuid4_info_regs *base)
 {
+	this_leaf->id = base->id;
+	this_leaf->attributes = CACHE_ID;
 	this_leaf->level = base->eax.split.level;
 	this_leaf->type = cache_type_map[base->eax.split.type];
 	this_leaf->coherency_line_size =
@@ -920,6 +923,22 @@ static int __init_cache_level(unsigned int cpu)
 	return 0;
 }
 
+/*
+ * The max shared threads number comes from CPUID.4:EAX[25-14] with input
+ * ECX as cache index. Then right shift apicid by the number's order to get
+ * cache id for this cache node.
+ */
+static void get_cache_id(int cpu, struct _cpuid4_info_regs *id4_regs)
+{
+	struct cpuinfo_x86 *c = &cpu_data(cpu);
+	unsigned long num_threads_sharing;
+	int index_msb;
+
+	num_threads_sharing = 1 + id4_regs->eax.split.num_threads_sharing;
+	index_msb = get_count_order(num_threads_sharing);
+	id4_regs->id = c->apicid >> index_msb;
+}
+
 static int __populate_cache_leaves(unsigned int cpu)
 {
 	unsigned int idx, ret;
@@ -931,6 +950,7 @@ static int __populate_cache_leaves(unsigned int cpu)
 		ret = cpuid4_cache_lookup_regs(idx, &id4_regs);
 		if (ret)
 			return ret;
+		get_cache_id(cpu, &id4_regs);
 		ci_leaf_init(this_leaf++, &id4_regs);
 		__cache_cpumap_setup(cpu, idx, &id4_regs);
 	}
-- 
2.5.0

[toc] | [prev] | [next] | [standalone]

#1478846 — [PATCH v2 09/33] x86/intel_rdt: Add L3 cache capacity bitmask management

From: Vikas Shivappa <vikas.shivappa@linux.intel.com>

This patch adds different APIs to manage the L3 cache capacity bitmask.
The capacity bit mask(CBM) needs to have only contiguous bits set. The
current implementation has a global CBM for each class of service id.
There are APIs added to update the CBM via MSR write to IA32_L3_MASK_n
on all packages. Other APIs are to read and write entries to the
clos_cbm_table.

Signed-off-by: Vikas Shivappa <vikas.shivappa@linux.intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Reviewed-by: Tony Luck <tony.luck@intel.com>
---
 arch/x86/include/asm/intel_rdt.h |  4 ++++
 arch/x86/kernel/cpu/intel_rdt.c  | 48 +++++++++++++++++++++++++++++++++++++++-
 2 files changed, 51 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/intel_rdt.h b/arch/x86/include/asm/intel_rdt.h
index 68bab26..68c9a79 100644
--- a/arch/x86/include/asm/intel_rdt.h
+++ b/arch/x86/include/asm/intel_rdt.h
@@ -3,6 +3,10 @@
 
 #ifdef CONFIG_INTEL_RDT
 
+#define MAX_CBM_LENGTH			32
+#define IA32_L3_CBM_BASE		0xc90
+#define CBM_FROM_INDEX(x)		(IA32_L3_CBM_BASE + x)
+
 struct clos_cbm_table {
 	unsigned long cbm;
 	unsigned int clos_refcnt;
diff --git a/arch/x86/kernel/cpu/intel_rdt.c b/arch/x86/kernel/cpu/intel_rdt.c
index b25940a..9cf3a7d 100644
--- a/arch/x86/kernel/cpu/intel_rdt.c
+++ b/arch/x86/kernel/cpu/intel_rdt.c
@@ -31,8 +31,22 @@ static struct clos_cbm_table *cctable;
  * closid availability bit map.
  */
 unsigned long *closmap;
+/*
+ * Mask of CPUs for writing CBM values. We only need one CPU per-socket.
+ */
+static cpumask_t rdt_cpumask;
+/*
+ * Temporary cpumask used during hot cpu notificaiton handling. The usage
+ * is serialized by hot cpu locks.
+ */
+static cpumask_t tmp_cpumask;
 static DEFINE_MUTEX(rdtgroup_mutex);
 
+struct rdt_remote_data {
+	int msr;
+	u64 val;
+};
+
 static inline void closid_get(u32 closid)
 {
 	struct clos_cbm_table *cct = &cctable[closid];
@@ -79,11 +93,41 @@ static void closid_put(u32 closid)
 		closid_free(closid);
 }
 
+static void msr_cpu_update(void *arg)
+{
+	struct rdt_remote_data *info = arg;
+
+	wrmsrl(info->msr, info->val);
+}
+
+/*
+ * msr_update_all() - Update the msr for all packages.
+ */
+static inline void msr_update_all(int msr, u64 val)
+{
+	struct rdt_remote_data info;
+
+	info.msr = msr;
+	info.val = val;
+	on_each_cpu_mask(&rdt_cpumask, msr_cpu_update, &info, 1);
+}
+
+static inline bool rdt_cpumask_update(int cpu)
+{
+	cpumask_and(&tmp_cpumask, &rdt_cpumask, topology_core_cpumask(cpu));
+	if (cpumask_empty(&tmp_cpumask)) {
+		cpumask_set_cpu(cpu, &rdt_cpumask);
+		return true;
+	}
+
+	return false;
+}
+
 static int __init intel_rdt_late_init(void)
 {
 	struct cpuinfo_x86 *c = &boot_cpu_data;
 	u32 maxid;
-	int err = 0, size;
+	int err = 0, size, i;
 
 	if (!cpu_has(c, X86_FEATURE_CAT_L3))
 		return -ENODEV;
@@ -105,6 +149,8 @@ static int __init intel_rdt_late_init(void)
 		goto out_err;
 	}
 
+	for_each_online_cpu(i)
+		rdt_cpumask_update(i);
 	pr_info("Intel cache allocation enabled\n");
 out_err:
 
-- 
2.5.0

[toc] | [prev] | [next] | [standalone]

#1478986 — Re: [PATCH v2 09/33] x86/intel_rdt: Add L3 cache capacity bitmask management

On Thu, 8 Sep 2016, Fenghua Yu wrote:
>  
> +	for_each_online_cpu(i)
> +		rdt_cpumask_update(i);

The only reason why this does not blow up in your face is that at this
point the secondary cpus have been brought up already and user space is not
yet running, so cpu hotplug cannot happen in parallel. Protection by chance
is never a good idea.

Thanks,

	tglx

[toc] | [prev] | [next] | [standalone]

#1481521 — Re: [PATCH v2 09/33] x86/intel_rdt: Add L3 cache capacity bitmask management

On 8 September 2016 at 04:57, Fenghua Yu <fenghua.yu@intel.com> wrote:
> diff --git a/arch/x86/kernel/cpu/intel_rdt.c b/arch/x86/kernel/cpu/intel_rdt.c
> index b25940a..9cf3a7d 100644
> --- a/arch/x86/kernel/cpu/intel_rdt.c
> +++ b/arch/x86/kernel/cpu/intel_rdt.c
> @@ -31,8 +31,22 @@ static struct clos_cbm_table *cctable;
>   * closid availability bit map.
>   */
>  unsigned long *closmap;
> +/*
> + * Mask of CPUs for writing CBM values. We only need one CPU per-socket.

Does the second line make sense here?

> + */
> +static cpumask_t rdt_cpumask;
> +/*
> + * Temporary cpumask used during hot cpu notificaiton handling. The usage
> + * is serialized by hot cpu locks.

s/notificaiton/notification


--
Nilay

[toc] | [prev] | [next] | [standalone]

#1478847 — [PATCH v2 07/33] x86/intel_rdt: Add support for Cache Allocation detection

From: Vikas Shivappa <vikas.shivappa@linux.intel.com>

This patch includes CPUID enumeration routines for Cache allocation and
new values to track resources to the cpuinfo_x86 structure.

Cache allocation provides a way for the Software (OS/VMM) to restrict
cache allocation to a defined 'subset' of cache which may be overlapping
with other 'subsets'. This feature is used when allocating a line in
cache ie when pulling new data into the cache. The programming of the
hardware is done via programming MSRs (model specific registers).

Signed-off-by: Vikas Shivappa <vikas.shivappa@linux.intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Reviewed-by: Tony Luck <tony.luck@intel.com>
---
 arch/x86/include/asm/cpufeature.h        |  8 +++++--
 arch/x86/include/asm/cpufeatures.h       |  6 +++++-
 arch/x86/include/asm/disabled-features.h |  3 ++-
 arch/x86/include/asm/processor.h         |  3 +++
 arch/x86/include/asm/required-features.h |  3 ++-
 arch/x86/kernel/cpu/common.c             | 19 ++++++++++++++++
 arch/x86/kernel/cpu/intel_rdt.c          | 37 ++++++++++++++++++++++++++++++++
 7 files changed, 74 insertions(+), 5 deletions(-)
 create mode 100644 arch/x86/kernel/cpu/intel_rdt.c

diff --git a/arch/x86/include/asm/cpufeature.h b/arch/x86/include/asm/cpufeature.h
index 1d2b69f..9985b4cf 100644
--- a/arch/x86/include/asm/cpufeature.h
+++ b/arch/x86/include/asm/cpufeature.h
@@ -28,6 +28,8 @@ enum cpuid_leafs
 	CPUID_8000_000A_EDX,
 	CPUID_7_ECX,
 	CPUID_8000_0007_EBX,
+	CPUID_10_0_EBX,
+	CPUID_10_1_ECX,
 };
 
 #ifdef CONFIG_X86_FEATURE_NAMES
@@ -78,8 +80,9 @@ extern const char * const x86_bug_flags[NBUGINTS*32];
 	   CHECK_BIT_IN_MASK_WORD(REQUIRED_MASK, 15, feature_bit) ||	\
 	   CHECK_BIT_IN_MASK_WORD(REQUIRED_MASK, 16, feature_bit) ||	\
 	   CHECK_BIT_IN_MASK_WORD(REQUIRED_MASK, 17, feature_bit) ||	\
+	   CHECK_BIT_IN_MASK_WORD(REQUIRED_MASK, 18, feature_bit) ||	\
 	   REQUIRED_MASK_CHECK					  ||	\
-	   BUILD_BUG_ON_ZERO(NCAPINTS != 18))
+	   BUILD_BUG_ON_ZERO(NCAPINTS != 19))
 
 #define DISABLED_MASK_BIT_SET(feature_bit)				\
 	 ( CHECK_BIT_IN_MASK_WORD(DISABLED_MASK,  0, feature_bit) ||	\
@@ -100,8 +103,9 @@ extern const char * const x86_bug_flags[NBUGINTS*32];
 	   CHECK_BIT_IN_MASK_WORD(DISABLED_MASK, 15, feature_bit) ||	\
 	   CHECK_BIT_IN_MASK_WORD(DISABLED_MASK, 16, feature_bit) ||	\
 	   CHECK_BIT_IN_MASK_WORD(DISABLED_MASK, 17, feature_bit) ||	\
+	   CHECK_BIT_IN_MASK_WORD(DISABLED_MASK, 18, feature_bit) ||	\
 	   DISABLED_MASK_CHECK					  ||	\
-	   BUILD_BUG_ON_ZERO(NCAPINTS != 18))
+	   BUILD_BUG_ON_ZERO(NCAPINTS != 19))
 
 #define cpu_has(c, bit)							\
 	(__builtin_constant_p(bit) && REQUIRED_MASK_BIT_SET(bit) ? 1 :	\
diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index 92a8308..62d979b9 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -12,7 +12,7 @@
 /*
  * Defines x86 CPU feature bits
  */
-#define NCAPINTS	18	/* N 32-bit words worth of info */
+#define NCAPINTS	19	/* N 32-bit words worth of info */
 #define NBUGINTS	1	/* N 32-bit bug flags */
 
 /*
@@ -220,6 +220,7 @@
 #define X86_FEATURE_RTM		( 9*32+11) /* Restricted Transactional Memory */
 #define X86_FEATURE_CQM		( 9*32+12) /* Cache QoS Monitoring */
 #define X86_FEATURE_MPX		( 9*32+14) /* Memory Protection Extension */
+#define X86_FEATURE_RDT		( 9*32+15) /* Resource Director Technology */
 #define X86_FEATURE_AVX512F	( 9*32+16) /* AVX-512 Foundation */
 #define X86_FEATURE_AVX512DQ	( 9*32+17) /* AVX-512 DQ (Double/Quad granular) Instructions */
 #define X86_FEATURE_RDSEED	( 9*32+18) /* The RDSEED instruction */
@@ -286,6 +287,9 @@
 #define X86_FEATURE_SUCCOR	(17*32+1) /* Uncorrectable error containment and recovery */
 #define X86_FEATURE_SMCA	(17*32+3) /* Scalable MCA */
 
+/* Intel-defined CPU features, CPUID level 0x00000010:0 (ebx), word 18 */
+#define X86_FEATURE_CAT_L3      (18*32+ 1) /* Cache Allocation L3 */
+
 /*
  * BUG word(s)
  */
diff --git a/arch/x86/include/asm/disabled-features.h b/arch/x86/include/asm/disabled-features.h
index 85599ad..8b45e08 100644
--- a/arch/x86/include/asm/disabled-features.h
+++ b/arch/x86/include/asm/disabled-features.h
@@ -57,6 +57,7 @@
 #define DISABLED_MASK15	0
 #define DISABLED_MASK16	(DISABLE_PKU|DISABLE_OSPKE)
 #define DISABLED_MASK17	0
-#define DISABLED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 18)
+#define DISABLED_MASK18	0
+#define DISABLED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 19)
 
 #endif /* _ASM_X86_DISABLED_FEATURES_H */
diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
index 63def95..e940b2d 100644
--- a/arch/x86/include/asm/processor.h
+++ b/arch/x86/include/asm/processor.h
@@ -119,6 +119,9 @@ struct cpuinfo_x86 {
 	int			x86_cache_occ_scale;	/* scale to bytes */
 	int			x86_power;
 	unsigned long		loops_per_jiffy;
+	/* Cache Allocation values: */
+	u16			x86_l3_max_cbm_len;
+	u16			x86_l3_max_closid;
 	/* cpuid returned max cores value: */
 	u16			 x86_max_cores;
 	u16			apicid;
diff --git a/arch/x86/include/asm/required-features.h b/arch/x86/include/asm/required-features.h
index fac9a5c..6847d85 100644
--- a/arch/x86/include/asm/required-features.h
+++ b/arch/x86/include/asm/required-features.h
@@ -100,6 +100,7 @@
 #define REQUIRED_MASK15	0
 #define REQUIRED_MASK16	0
 #define REQUIRED_MASK17	0
-#define REQUIRED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 18)
+#define REQUIRED_MASK18	0
+#define REQUIRED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 19)
 
 #endif /* _ASM_X86_REQUIRED_FEATURES_H */
diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index 809eda0..997d1d5 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -711,6 +711,25 @@ void get_cpu_cap(struct cpuinfo_x86 *c)
 		}
 	}
 
+	/* Additional Intel-defined flags: level 0x00000010 */
+	if (c->cpuid_level >= 0x00000010) {
+		u32 eax, ebx, ecx, edx;
+
+		cpuid_count(0x00000010, 0, &eax, &ebx, &ecx, &edx);
+		c->x86_capability[CPUID_10_0_EBX] = ebx;
+
+		if (cpu_has(c, X86_FEATURE_CAT_L3)) {
+
+			cpuid_count(0x00000010, 1, &eax, &ebx, &ecx, &edx);
+			c->x86_l3_max_closid = edx + 1;
+			c->x86_l3_max_cbm_len = eax + 1;
+			c->x86_capability[CPUID_10_1_ECX] = ecx;
+		} else {
+			c->x86_l3_max_closid = -1;
+			c->x86_l3_max_cbm_len = -1;
+		}
+	}
+
 	/* AMD-defined flags: level 0x80000001 */
 	eax = cpuid_eax(0x80000000);
 	c->extended_cpuid_level = eax;
diff --git a/arch/x86/kernel/cpu/intel_rdt.c b/arch/x86/kernel/cpu/intel_rdt.c
new file mode 100644
index 0000000..fcd0642
--- /dev/null
+++ b/arch/x86/kernel/cpu/intel_rdt.c
@@ -0,0 +1,37 @@
+/*
+ * Resource Director Technology(RDT)
+ * - Cache Allocation code.
+ *
+ * Copyright (C) 2014 Intel Corporation
+ *
+ * 2015-05-25 Written by
+ *    Vikas Shivappa <vikas.shivappa@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * More information about RDT be found in the Intel (R) x86 Architecture
+ * Software Developer Manual June 2015, volume 3, section 17.15.
+ */
+#include <linux/slab.h>
+#include <linux/err.h>
+
+static int __init intel_rdt_late_init(void)
+{
+	struct cpuinfo_x86 *c = &boot_cpu_data;
+
+	if (!cpu_has(c, X86_FEATURE_CAT_L3))
+		return -ENODEV;
+
+	pr_info("Intel cache allocation detected\n");
+
+	return 0;
+}
+
+late_initcall(intel_rdt_late_init);
-- 
2.5.0

[toc] | [prev] | [next] | [standalone]

#1479143 — Re: [PATCH v2 07/33] x86/intel_rdt: Add support for Cache Allocation detection

On Thu, Sep 08, 2016 at 02:57:01AM -0700, Fenghua Yu wrote:
> From: Vikas Shivappa <vikas.shivappa@linux.intel.com>
> 
> This patch includes CPUID enumeration routines for Cache allocation and
> new values to track resources to the cpuinfo_x86 structure.
> 
> Cache allocation provides a way for the Software (OS/VMM) to restrict
> cache allocation to a defined 'subset' of cache which may be overlapping
> with other 'subsets'. This feature is used when allocating a line in
> cache ie when pulling new data into the cache. The programming of the
> hardware is done via programming MSRs (model specific registers).
> 
> Signed-off-by: Vikas Shivappa <vikas.shivappa@linux.intel.com>
> Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
> Reviewed-by: Tony Luck <tony.luck@intel.com>
> ---

...

> diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
> index 92a8308..62d979b9 100644
> --- a/arch/x86/include/asm/cpufeatures.h
> +++ b/arch/x86/include/asm/cpufeatures.h
> @@ -12,7 +12,7 @@
>  /*
>   * Defines x86 CPU feature bits
>   */
> -#define NCAPINTS	18	/* N 32-bit words worth of info */
> +#define NCAPINTS	19	/* N 32-bit words worth of info */
>  #define NBUGINTS	1	/* N 32-bit bug flags */
>  
>  /*
> @@ -220,6 +220,7 @@
>  #define X86_FEATURE_RTM		( 9*32+11) /* Restricted Transactional Memory */
>  #define X86_FEATURE_CQM		( 9*32+12) /* Cache QoS Monitoring */
>  #define X86_FEATURE_MPX		( 9*32+14) /* Memory Protection Extension */
> +#define X86_FEATURE_RDT		( 9*32+15) /* Resource Director Technology */
>  #define X86_FEATURE_AVX512F	( 9*32+16) /* AVX-512 Foundation */
>  #define X86_FEATURE_AVX512DQ	( 9*32+17) /* AVX-512 DQ (Double/Quad granular) Instructions */
>  #define X86_FEATURE_RDSEED	( 9*32+18) /* The RDSEED instruction */
> @@ -286,6 +287,9 @@
>  #define X86_FEATURE_SUCCOR	(17*32+1) /* Uncorrectable error containment and recovery */
>  #define X86_FEATURE_SMCA	(17*32+3) /* Scalable MCA */
>  
> +/* Intel-defined CPU features, CPUID level 0x00000010:0 (ebx), word 18 */

Seems like this leaf is dedicated to CAT and has only 2 feature bits
defined in the SDM. Please use init_scattered_cpuid_features() instead
of adding a whole CAP word.

-- 
Regards/Gruss,
    Boris.

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
-- 

[toc] | [prev] | [next] | [standalone]

#1479379 — RE: [PATCH v2 07/33] x86/intel_rdt: Add support for Cache Allocation detection

> From: Borislav Petkov [mailto:bp@suse.de]
> On Thu, Sep 08, 2016 at 02:57:01AM -0700, Fenghua Yu wrote:
> > From: Vikas Shivappa <vikas.shivappa@linux.intel.com>
> >
> > This patch includes CPUID enumeration routines for Cache allocation
> > and new values to track resources to the cpuinfo_x86 structure.
> >
> > Cache allocation provides a way for the Software (OS/VMM) to restrict
> > cache allocation to a defined 'subset' of cache which may be
> > overlapping with other 'subsets'. This feature is used when allocating
> > a line in cache ie when pulling new data into the cache. The
> > programming of the hardware is done via programming MSRs (model
> specific registers).
> >
> > Signed-off-by: Vikas Shivappa <vikas.shivappa@linux.intel.com>
> > Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
> > Reviewed-by: Tony Luck <tony.luck@intel.com>
> > ---
> 
> ...
> 
> > diff --git a/arch/x86/include/asm/cpufeatures.h
> > b/arch/x86/include/asm/cpufeatures.h
> > index 92a8308..62d979b9 100644
> > --- a/arch/x86/include/asm/cpufeatures.h
> > +++ b/arch/x86/include/asm/cpufeatures.h
> > @@ -12,7 +12,7 @@
> >  /*
> >   * Defines x86 CPU feature bits
> >   */
> > -#define NCAPINTS	18	/* N 32-bit words worth of info */
> > +#define NCAPINTS	19	/* N 32-bit words worth of info */
> >  #define NBUGINTS	1	/* N 32-bit bug flags */
> >
> >  /*
> > @@ -220,6 +220,7 @@
> >  #define X86_FEATURE_RTM		( 9*32+11) /* Restricted Transactional
> Memory */
> >  #define X86_FEATURE_CQM		( 9*32+12) /* Cache QoS Monitoring
> */
> >  #define X86_FEATURE_MPX		( 9*32+14) /* Memory Protection
> Extension */
> > +#define X86_FEATURE_RDT		( 9*32+15) /* Resource Director
> Technology */
> >  #define X86_FEATURE_AVX512F	( 9*32+16) /* AVX-512 Foundation */
> >  #define X86_FEATURE_AVX512DQ	( 9*32+17) /* AVX-512 DQ
> (Double/Quad granular) Instructions */
> >  #define X86_FEATURE_RDSEED	( 9*32+18) /* The RDSEED instruction
> */
> > @@ -286,6 +287,9 @@
> >  #define X86_FEATURE_SUCCOR	(17*32+1) /* Uncorrectable error
> containment and recovery */
> >  #define X86_FEATURE_SMCA	(17*32+3) /* Scalable MCA */
> >
> > +/* Intel-defined CPU features, CPUID level 0x00000010:0 (ebx), word
> > +18 */
> 
> Seems like this leaf is dedicated to CAT and has only 2 feature bits defined in
> the SDM. Please use init_scattered_cpuid_features() instead of adding a
> whole CAP word.

Actually this leaf will be extended to have more bits for more resources allocation.

Thanks.

-Fenghua

[toc] | [prev] | [next] | [standalone]

#1479392 — Re: [PATCH v2 07/33] x86/intel_rdt: Add support for Cache Allocation detection

On Thu, Sep 08, 2016 at 04:53:52PM +0000, Yu, Fenghua wrote:
> Actually this leaf will be extended to have more bits for more
> resources allocation.

You can move it to a separate leaf then.

-- 
Regards/Gruss,
    Boris.

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
-- 

[toc] | [prev] | [next] | [standalone]

#1479218 — Re: [PATCH v2 07/33] x86/intel_rdt: Add support for Cache Allocation detection

On Thu, 8 Sep 2016, Fenghua Yu wrote:
> +			cpuid_count(0x00000010, 1, &eax, &ebx, &ecx, &edx);
> +			c->x86_l3_max_closid = edx + 1;
> +			c->x86_l3_max_cbm_len = eax + 1;

According to the SDM:

EAX     Bits  4:0:  Length of the capacity bit mask for the corresponding ResID.
        Bits 31:05: Reserved

EDX	Bits 15:0:  Highest COS number supported for this ResID.
	Bits 31:16: Reserved

So why are we assuming that bits 31-5 of EAX and 16-31 of EDX are going to
be zero forever and if not that they are just extending the existing bits?
If that's the case then we don't need to mask out the upper bits, but the
code wants a proper comment about this.

Thanks,

	tglx

[toc] | [prev] | [next] | [standalone]

#1479253 — RE: [PATCH v2 07/33] x86/intel_rdt: Add support for Cache Allocation detection

> On Thu, 8 Sep 2016, Fenghua Yu wrote:
> > +			cpuid_count(0x00000010, 1, &eax, &ebx, &ecx,
> &edx);
> > +			c->x86_l3_max_closid = edx + 1;
> > +			c->x86_l3_max_cbm_len = eax + 1;
> 
> According to the SDM:
> 
> EAX     Bits  4:0:  Length of the capacity bit mask for the corresponding ResID.
>         Bits 31:05: Reserved
> 
> EDX	Bits 15:0:  Highest COS number supported for this ResID.
> 	Bits 31:16: Reserved
> 
> So why are we assuming that bits 31-5 of EAX and 16-31 of EDX are going to
> be zero forever and if not that they are just extending the existing bits?
> If that's the case then we don't need to mask out the upper bits, but the
> code wants a proper comment about this.

You are right. We cannot assume the upper bits are always zero. I fixed the
issue by masking out the upper bits.

Thanks.

-Fenghua 

[toc] | [prev] | [next] | [standalone]

Page 3 of 5 — ← Prev page 1 2 [3] 4 5  Next page →

Back to top | Article view | linux.kernel

csiph-web