Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1441783 > unrolled thread

[PATCH 00/32] Enable Intel Resource Allocation in Resource Director Technology

Back to article view | Back to linux.kernel

Contents

  [PATCH 00/32] Enable Intel Resource Allocation in Resource Director Technology  "Fenghua Yu" <fenghua.yu@intel.com> - 2016-07-13 00:10 +0200
    [PATCH 06/32] x86/intel_rdt: Hot cpu support for Cache Allocation "Fenghua Yu" <fenghua.yu@intel.com> - 2016-07-13 00:10 +0200
      Re: [PATCH 06/32] x86/intel_rdt: Hot cpu support for Cache  Allocation Thomas Gleixner <tglx@linutronix.de> - 2016-07-13 11:30 +0200
        Re: [PATCH 06/32] x86/intel_rdt: Hot cpu support for Cache  Allocation Shivappa Vikas <vikas.shivappa@linux.intel.com> - 2016-07-21 21:50 +0200
      Re: [PATCH 06/32] x86/intel_rdt: Hot cpu support for Cache Allocation David Carrillo-Cisneros <davidcc@google.com> - 2016-07-14 02:50 +0200
        RE: [PATCH 06/32] x86/intel_rdt: Hot cpu support for Cache  Allocation "Yu, Fenghua" <fenghua.yu@intel.com> - 2016-07-15 01:00 +0200
    [PATCH 19/32] sched.h: Add rg_list and rdtgroup in task_struct "Fenghua Yu" <fenghua.yu@intel.com> - 2016-07-13 00:10 +0200
      Re: [PATCH 19/32] sched.h: Add rg_list and rdtgroup in task_struct Thomas Gleixner <tglx@linutronix.de> - 2016-07-13 15:10 +0200
        RE: [PATCH 19/32] sched.h: Add rg_list and rdtgroup in task_struct "Yu, Fenghua" <fenghua.yu@intel.com> - 2016-07-13 20:00 +0200
      Re: [PATCH 19/32] sched.h: Add rg_list and rdtgroup in task_struct Nilay Vaish <nilayvaish@gmail.com> - 2016-07-28 08:00 +0200
    [PATCH 21/32] x86/intel_rdt.h: Header for inter_rdt.c "Fenghua Yu" <fenghua.yu@intel.com> - 2016-07-13 00:10 +0200
      Re: [PATCH 21/32] x86/intel_rdt.h: Header for inter_rdt.c Nilay Vaish <nilayvaish@gmail.com> - 2016-07-28 16:10 +0200
    [PATCH 12/32] x86/intel_rdt: Hot cpu update for code data prioritization "Fenghua Yu" <fenghua.yu@intel.com> - 2016-07-13 00:10 +0200
    [PATCH 27/32] x86/intel_rdt_rdtgroup.c: Implement rscctrl file system commands "Fenghua Yu" <fenghua.yu@intel.com> - 2016-07-13 00:10 +0200
    [PATCH 30/32] x86/intel_rdt_rdtgroup.c: Process schemas input from rscctrl interface "Fenghua Yu" <fenghua.yu@intel.com> - 2016-07-13 00:10 +0200
      Re: [PATCH 30/32] x86/intel_rdt_rdtgroup.c: Process schemas input  from rscctrl interface David Carrillo-Cisneros <davidcc@google.com> - 2016-07-14 02:50 +0200
        RE: [PATCH 30/32] x86/intel_rdt_rdtgroup.c: Process schemas input  from rscctrl interface "Yu, Fenghua" <fenghua.yu@intel.com> - 2016-07-14 08:20 +0200
        Re: [PATCH 30/32] x86/intel_rdt_rdtgroup.c: Process schemas input  from rscctrl interface Thomas Gleixner <tglx@linutronix.de> - 2016-07-14 08:20 +0200
        RE: [PATCH 30/32] x86/intel_rdt_rdtgroup.c: Process schemas input  from rscctrl interface "Yu, Fenghua" <fenghua.yu@intel.com> - 2016-07-14 08:40 +0200
    [PATCH 02/32] x86/intel_rdt: Add support for Cache Allocation detection "Fenghua Yu" <fenghua.yu@intel.com> - 2016-07-13 00:10 +0200
      Re: [PATCH 02/32] x86/intel_rdt: Add support for Cache Allocation detection Nilay Vaish <nilayvaish@gmail.com> - 2016-07-26 21:10 +0200
    [PATCH 01/32] x86/intel_rdt: Cache Allocation documentation "Fenghua Yu" <fenghua.yu@intel.com> - 2016-07-13 00:10 +0200
    [PATCH 28/32] x86/intel_rdt_rdtgroup.c: Read and write cpus "Fenghua Yu" <fenghua.yu@intel.com> - 2016-07-13 00:10 +0200
    [PATCH 03/32] x86/intel_rdt: Add Class of service management "Fenghua Yu" <fenghua.yu@intel.com> - 2016-07-13 00:20 +0200
    [PATCH 04/32] x86/intel_rdt: Add L3 cache capacity bitmask management "Fenghua Yu" <fenghua.yu@intel.com> - 2016-07-13 00:20 +0200
      Re: [PATCH 04/32] x86/intel_rdt: Add L3 cache capacity bitmask  management Marcelo Tosatti <mtosatti@redhat.com> - 2016-07-22 23:10 +0200
        Re: [PATCH 04/32] x86/intel_rdt: Add L3 cache capacity bitmask  management "Luck, Tony" <tony.luck@intel.com> - 2016-07-22 23:50 +0200
          Re: [PATCH 04/32] x86/intel_rdt: Add L3 cache capacity bitmask  management Marcelo Tosatti <mtosatti@redhat.com> - 2016-07-26 04:20 +0200
            Re: [PATCH 04/32] x86/intel_rdt: Add L3 cache capacity bitmask  management "Luck, Tony" <tony.luck@intel.com> - 2016-07-26 05:20 +0200
            Re: [PATCH 04/32] x86/intel_rdt: Add L3 cache capacity bitmask  management Shivappa Vikas <vikas.shivappa@linux.intel.com> - 2016-07-26 19:10 +0200
    [PATCH 05/32] x86/intel_rdt: Implement scheduling support for Intel RDT "Fenghua Yu" <fenghua.yu@intel.com> - 2016-07-13 00:20 +0200
      Re: [PATCH 05/32] x86/intel_rdt: Implement scheduling support for  Intel RDT Nilay Vaish <nilayvaish@gmail.com> - 2016-07-25 18:30 +0200
      Re: [PATCH 05/32] x86/intel_rdt: Implement scheduling support for  Intel RDT Nilay Vaish <nilayvaish@gmail.com> - 2016-07-25 18:40 +0200
        Re: [PATCH 05/32] x86/intel_rdt: Implement scheduling support for  Intel RDT "Luck, Tony" <tony.luck@intel.com> - 2016-07-25 20:10 +0200
          Re: [PATCH 05/32] x86/intel_rdt: Implement scheduling support for  Intel RDT David Carrillo-Cisneros <davidcc@google.com> - 2016-07-26 00:50 +0200

Page 2 of 2 — ← Prev page 1 [2]

#1450798 — Re: [PATCH 02/32] x86/intel_rdt: Add support for Cache Allocation detection

On 12 July 2016 at 20:02, Fenghua Yu <fenghua.yu@intel.com> wrote:
> diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
> index 0fe6953..42c90cb 100644
> --- a/arch/x86/kernel/cpu/common.c
> +++ b/arch/x86/kernel/cpu/common.c
> @@ -711,6 +711,21 @@ void get_cpu_cap(struct cpuinfo_x86 *c)
>                 }
>         }
>
> +       /* Additional Intel-defined flags: level 0x00000010 */
> +       if (c->cpuid_level >= 0x00000010) {
> +               u32 eax, ebx, ecx, edx;
> +
> +               cpuid_count(0x00000010, 0, &eax, &ebx, &ecx, &edx);
> +               c->x86_capability[14] = ebx;

Should we have a name for this capability / leaf, instead of that
constant 14?  grep informs me that everywhere else we use enum of type
cpuid_leafs.


--
Nilay

[toc] | [prev] | [next] | [standalone]

#1441791 — [PATCH 01/32] x86/intel_rdt: Cache Allocation documentation

From: Vikas Shivappa <vikas.shivappa@linux.intel.com>

Adds a description of Cache allocation technology, overview of kernel
framework implementation. The framework has APIs to manage class of
service, capacity bitmask(CBM), scheduling support and other
architecture specific implementation. The APIs are used to build the
rscctrl interface in later patches.

Cache allocation is a sub-feature of Resource Director Technology (RDT)
or Platform Shared resource control which provides support to control
Platform shared resources like L3 cache.

Cache Allocation Technology provides a way for the Software (OS/VMM) to
restrict cache allocation to a defined 'subset' of cache which may be
overlapping with other 'subsets'. This feature is used when allocating a
line in cache ie when pulling new data into the cache. The tasks are
grouped into CLOS (class of service). OS uses MSR writes to indicate the
CLOSid of the thread when scheduling in and to indicate the cache
capacity associated with the CLOSid. Currently cache allocation is
supported for L3 cache.

More information can be found in the Intel SDM June 2015, Volume 3,
section 17.16.

Signed-off-by: Vikas Shivappa <vikas.shivappa@linux.intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Reviewed-by: Tony Luck <tony.luck@intel.com>
---
 Documentation/x86/intel_rdt.txt | 109 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 109 insertions(+)
 create mode 100644 Documentation/x86/intel_rdt.txt

diff --git a/Documentation/x86/intel_rdt.txt b/Documentation/x86/intel_rdt.txt
new file mode 100644
index 0000000..05ec819
--- /dev/null
+++ b/Documentation/x86/intel_rdt.txt
@@ -0,0 +1,109 @@
+        Intel RDT
+        ---------
+
+Copyright (C) 2014 Intel Corporation
+Written by vikas.shivappa@linux.intel.com
+
+CONTENTS:
+=========
+
+1. Cache Allocation Technology
+  1.1 What is RDT and Cache allocation ?
+  1.2 Why is Cache allocation needed ?
+  1.3 Cache allocation implementation overview
+  1.4 Assignment of CBM and CLOS
+  1.5 Scheduling and Context Switch
+
+1. Cache Allocation Technology
+===================================
+
+1.1 What is RDT and Cache allocation
+------------------------------------
+
+Cache allocation is a sub-feature of Resource Director Technology (RDT)
+Allocation or Platform Shared resource control which provides support to
+control Platform shared resources like L3 cache. Currently L3 Cache is
+the only resource that is supported in RDT. More information can be
+found in the Intel SDM June 2015, Volume 3, section 17.16.
+
+Cache Allocation Technology provides a way for the Software (OS/VMM) to
+restrict cache allocation to a defined 'subset' of cache which may be
+overlapping with other 'subsets'. This feature is used when allocating a
+line in cache ie when pulling new data into the cache. The programming
+of the h/w is done via programming MSRs.
+
+The different cache subsets are identified by CLOS identifier (class of
+service) and each CLOS has a CBM (cache bit mask). The CBM is a
+contiguous set of bits which defines the amount of cache resource that
+is available for each 'subset'.
+
+1.2 Why is Cache allocation needed
+----------------------------------
+
+In todays new processors the number of cores is continuously increasing
+especially in large scale usage models where VMs are used like
+webservers and datacenters. The number of cores increase the number of
+threads or workloads that can simultaneously be run. When
+multi-threaded-applications, VMs, workloads run concurrently they
+compete for shared resources including L3 cache.
+
+The architecture also allows dynamically changing these subsets during
+runtime to further optimize the performance of the higher priority
+application with minimal degradation to the low priority app.
+Additionally, resources can be rebalanced for system throughput benefit.
+
+This technique may be useful in managing large computer server systems
+with large L3 cache, in the cloud and container context. Examples may be
+large servers running instances of webservers or database servers. In
+such complex systems, these subsets can be used for more careful placing
+of the available cache resources by a centralized root accessible
+interface.
+
+A specific use case may be to solve the noisy neighbour issue when a app
+which is constantly copying data like streaming app is using large
+amount of cache which could have otherwise been used by a high priority
+computing application. Using the cache allocation feature, the streaming
+application can be confined to use a smaller cache and the high priority
+application be awarded a larger amount of cache space.
+
+1.3 Cache allocation implementation Overview
+--------------------------------------------
+
+Kernel has a new field in the task_struct called 'closid' which
+represents the Class of service ID of the task.
+
+There is a 1:1 CLOSid <-> CBM (capacity bit mask) mapping. A CLOS (Class
+of service) is represented by a CLOSid. Each closid would have one CBM
+and would just represent one cache 'subset'.  The tasks would get to
+fill the L3 cache represented by the capacity bit mask or CBM.
+
+The APIs to manage the closid and CBM can be used to develop user
+interfaces.
+
+1.4 Assignment of CBM, CLOS
+---------------------------
+
+The framework provides APIs to manage the closid and CBM which can be
+used to develop user/kernel mode interfaces.
+
+1.5 Scheduling and Context Switch
+---------------------------------
+
+During context switch kernel implements this by writing the CLOSid of
+the task to the CPU's IA32_PQR_ASSOC MSR. The MSR is only written when
+there is a change in the CLOSid for the CPU in order to minimize the
+latency incurred during context switch.
+
+The following considerations are done for the PQR MSR write so that it
+has minimal impact on scheduling hot path:
+ - This path doesn't exist on any non-intel platforms.
+ - On Intel platforms, this would not exist by default unless INTEL_RDT
+ is enabled.
+ - remains a no-op when INTEL_RDT is enabled and intel hardware does
+ not support the feature.
+ - When feature is available, does not do MSR write till the user
+ starts using the feature *and* assigns a new cache capacity mask.
+ - per cpu PQR values are cached and the MSR write is only done when
+ there is a task with different PQR is scheduled on the CPU. Typically
+ if the task groups are bound to be scheduled on a set of CPUs, the
+ number of MSR writes is greatly reduced.
-- 
2.5.0

[toc] | [prev] | [next] | [standalone]

#1441792 — [PATCH 28/32] x86/intel_rdt_rdtgroup.c: Read and write cpus

From: Fenghua Yu <fenghua.yu@intel.com>

Normally each task is associated with one rdtgroup and we use the schema
for that rdtgroup whenever the task is running. The user can designate
some cpus to always use the same schema, regardless of which task is
running. To do that the user write a cpumask bit string to the "cpus"
file.

A cpu can only be listed in one rdtgroup. If the user specifies a cpu
that is currently assigned to a different rdtgroup, it is removed
from that rdtgroup.

See Documentation/x86/intel_rdt_ui.txt

Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Reviewed-by: Tony Luck <tony.luck@intel.com>
---
 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 54 ++++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)

diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
index 91ea3509..b5f42f5 100644
--- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
+++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
@@ -767,6 +767,60 @@ void rdtgroup_exit(struct task_struct *tsk)
 
 static struct rdtgroup *rdtgroup_kn_lock_live(struct kernfs_node *kn);
 static void rdtgroup_kn_unlock(struct kernfs_node *kn);
+static int rdtgroup_cpus_show(struct seq_file *s, void *v)
+{
+	struct kernfs_open_file *of = s->private;
+	struct rdtgroup *rdtgrp;
+
+	rdtgrp = rdtgroup_kn_lock_live(of->kn);
+	seq_printf(s, "%*pb\n", cpumask_pr_args(&rdtgrp->cpu_mask));
+	rdtgroup_kn_unlock(of->kn);
+
+	return 0;
+}
+
+static ssize_t rdtgroup_cpus_write(struct kernfs_open_file *of,
+			char *buf, size_t nbytes, loff_t off)
+{
+	struct rdtgroup *rdtgrp;
+	unsigned long bitmap[BITS_TO_LONGS(NR_CPUS)];
+	struct cpumask *cpumask;
+	int cpu;
+	struct list_head *l;
+	struct rdtgroup *r;
+
+	if (!buf)
+		return -EINVAL;
+
+	rdtgrp = rdtgroup_kn_lock_live(of->kn);
+	if (!rdtgrp)
+		return -ENODEV;
+
+	if (list_empty(&rdtgroup_lists))
+		goto end;
+
+	__bitmap_parse(buf, strlen(buf), 0, bitmap, nr_cpu_ids);
+
+	cpumask = to_cpumask(bitmap);
+
+	list_for_each(l, &rdtgroup_lists) {
+		r = list_entry(l, struct rdtgroup, rdtgroup_list);
+		if (r == rdtgrp)
+			continue;
+
+		for_each_cpu_and(cpu, &r->cpu_mask, cpumask)
+			cpumask_clear_cpu(cpu, &r->cpu_mask);
+	}
+
+	cpumask_copy(&rdtgrp->cpu_mask, cpumask);
+	for_each_cpu(cpu, cpumask)
+		per_cpu(cpu_rdtgroup, cpu) = rdtgrp;
+
+end:
+	rdtgroup_kn_unlock(of->kn);
+
+	return nbytes;
+}
 
 static struct rftype rdtgroup_partition_base_files[] = {
 	{
-- 
2.5.0

[toc] | [prev] | [next] | [standalone]

#1441794 — [PATCH 03/32] x86/intel_rdt: Add Class of service management

From: Vikas Shivappa <vikas.shivappa@linux.intel.com>

Adds some data-structures and APIs to support Class of service
management(closid). There is a new clos_cbm table which keeps a 1:1
mapping between closid and capacity bit mask (cbm)
and a count of usage of closid. Each task would be associated with a
Closid at a time and this patch adds a new field closid to task_struct
to keep track of the same.

Signed-off-by: Vikas Shivappa <vikas.shivappa@linux.intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Reviewed-by: Tony Luck <tony.luck@intel.com>
---
 arch/x86/include/asm/intel_rdt.h | 12 ++++++
 arch/x86/kernel/cpu/intel_rdt.c  | 82 +++++++++++++++++++++++++++++++++++++++-
 include/linux/sched.h            |  3 ++
 3 files changed, 95 insertions(+), 2 deletions(-)
 create mode 100644 arch/x86/include/asm/intel_rdt.h

diff --git a/arch/x86/include/asm/intel_rdt.h b/arch/x86/include/asm/intel_rdt.h
new file mode 100644
index 0000000..88b7643
--- /dev/null
+++ b/arch/x86/include/asm/intel_rdt.h
@@ -0,0 +1,12 @@
+#ifndef _RDT_H_
+#define _RDT_H_
+
+#ifdef CONFIG_INTEL_RDT
+
+struct clos_cbm_table {
+	unsigned long l3_cbm;
+	unsigned int clos_refcnt;
+};
+
+#endif
+#endif
diff --git a/arch/x86/kernel/cpu/intel_rdt.c b/arch/x86/kernel/cpu/intel_rdt.c
index f49e970..d79213a 100644
--- a/arch/x86/kernel/cpu/intel_rdt.c
+++ b/arch/x86/kernel/cpu/intel_rdt.c
@@ -24,17 +24,95 @@
 
 #include <linux/slab.h>
 #include <linux/err.h>
+#include <asm/intel_rdt.h>
+
+/*
+ * cctable maintains 1:1 mapping between CLOSid and cache bitmask.
+ */
+static struct clos_cbm_table *cctable;
+/*
+ * closid availability bit map.
+ */
+unsigned long *closmap;
+static DEFINE_MUTEX(rdt_group_mutex);
+
+static inline void closid_get(u32 closid)
+{
+	struct clos_cbm_table *cct = &cctable[closid];
+
+	lockdep_assert_held(&rdt_group_mutex);
+
+	cct->clos_refcnt++;
+}
+
+static int closid_alloc(u32 *closid)
+{
+	u32 maxid;
+	u32 id;
+
+	lockdep_assert_held(&rdt_group_mutex);
+
+	maxid = boot_cpu_data.x86_cache_max_closid;
+	id = find_first_zero_bit(closmap, maxid);
+	if (id == maxid)
+		return -ENOSPC;
+
+	set_bit(id, closmap);
+	closid_get(id);
+	*closid = id;
+
+	return 0;
+}
+
+static inline void closid_free(u32 closid)
+{
+	clear_bit(closid, closmap);
+	cctable[closid].l3_cbm = 0;
+}
+
+static void closid_put(u32 closid)
+{
+	struct clos_cbm_table *cct = &cctable[closid];
+
+	lockdep_assert_held(&rdt_group_mutex);
+	if (WARN_ON(!cct->clos_refcnt))
+		return;
+
+	if (!--cct->clos_refcnt)
+		closid_free(closid);
+}
 
 static int __init intel_rdt_late_init(void)
 {
 	struct cpuinfo_x86 *c = &boot_cpu_data;
+	u32 maxid, max_cbm_len;
+	int err = 0, size;
 
 	if (!cpu_has(c, X86_FEATURE_CAT_L3))
 		return -ENODEV;
 
-	pr_info("Intel cache allocation detected\n");
+	maxid = c->x86_cache_max_closid;
+	max_cbm_len = c->x86_cache_max_cbm_len;
 
-	return 0;
+	size = maxid * sizeof(struct clos_cbm_table);
+	cctable = kzalloc(size, GFP_KERNEL);
+	if (!cctable) {
+		err = -ENOMEM;
+		goto out_err;
+	}
+
+	size = BITS_TO_LONGS(maxid) * sizeof(long);
+	closmap = kzalloc(size, GFP_KERNEL);
+	if (!closmap) {
+		kfree(cctable);
+		err = -ENOMEM;
+		goto out_err;
+	}
+
+	pr_info("Intel cache allocation enabled\n");
+out_err:
+
+	return err;
 }
 
 late_initcall(intel_rdt_late_init);
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 253538f..6005bba 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -1761,6 +1761,9 @@ struct task_struct {
 	/* cg_list protected by css_set_lock and tsk->alloc_lock */
 	struct list_head cg_list;
 #endif
+#ifdef CONFIG_INTEL_RDT
+	u32 closid;
+#endif
 #ifdef CONFIG_FUTEX
 	struct robust_list_head __user *robust_list;
 #ifdef CONFIG_COMPAT
-- 
2.5.0

[toc] | [prev] | [next] | [standalone]

#1441800 — [PATCH 04/32] x86/intel_rdt: Add L3 cache capacity bitmask management

From: Vikas Shivappa <vikas.shivappa@linux.intel.com>

This patch adds different APIs to manage the L3 cache capacity bitmask.
The capacity bit mask(CBM) needs to have only contiguous bits set. The
current implementation has a global CBM for each class of service id.
There are APIs added to update the CBM via MSR write to IA32_L3_MASK_n
on all packages. Other APIs are to read and write entries to the
clos_cbm_table.

Signed-off-by: Vikas Shivappa <vikas.shivappa@linux.intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Reviewed-by: Tony Luck <tony.luck@intel.com>
---
 arch/x86/include/asm/intel_rdt.h |   4 ++
 arch/x86/kernel/cpu/intel_rdt.c  | 133 ++++++++++++++++++++++++++++++++++++++-
 2 files changed, 136 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/intel_rdt.h b/arch/x86/include/asm/intel_rdt.h
index 88b7643..4f45dc8 100644
--- a/arch/x86/include/asm/intel_rdt.h
+++ b/arch/x86/include/asm/intel_rdt.h
@@ -3,6 +3,10 @@
 
 #ifdef CONFIG_INTEL_RDT
 
+#define MAX_CBM_LENGTH			32
+#define IA32_L3_CBM_BASE		0xc90
+#define CBM_FROM_INDEX(x)		(IA32_L3_CBM_BASE + x)
+
 struct clos_cbm_table {
 	unsigned long l3_cbm;
 	unsigned int clos_refcnt;
diff --git a/arch/x86/kernel/cpu/intel_rdt.c b/arch/x86/kernel/cpu/intel_rdt.c
index d79213a..6ad5b48 100644
--- a/arch/x86/kernel/cpu/intel_rdt.c
+++ b/arch/x86/kernel/cpu/intel_rdt.c
@@ -34,8 +34,22 @@ static struct clos_cbm_table *cctable;
  * closid availability bit map.
  */
 unsigned long *closmap;
+/*
+ * Mask of CPUs for writing CBM values. We only need one CPU per-socket.
+ */
+static cpumask_t rdt_cpumask;
+/*
+ * Temporary cpumask used during hot cpu notificaiton handling. The usage
+ * is serialized by hot cpu locks.
+ */
+static cpumask_t tmp_cpumask;
 static DEFINE_MUTEX(rdt_group_mutex);
 
+struct rdt_remote_data {
+	int msr;
+	u64 val;
+};
+
 static inline void closid_get(u32 closid)
 {
 	struct clos_cbm_table *cct = &cctable[closid];
@@ -82,11 +96,126 @@ static void closid_put(u32 closid)
 		closid_free(closid);
 }
 
+static bool cbm_validate(unsigned long var)
+{
+	u32 max_cbm_len = boot_cpu_data.x86_cache_max_cbm_len;
+	unsigned long first_bit, zero_bit;
+	u64 max_cbm;
+
+	if (bitmap_weight(&var, max_cbm_len) < 1)
+		return false;
+
+	max_cbm = (1ULL << max_cbm_len) - 1;
+	if (var & ~max_cbm)
+		return false;
+
+	first_bit = find_first_bit(&var, max_cbm_len);
+	zero_bit = find_next_zero_bit(&var, max_cbm_len, first_bit);
+
+	if (find_next_bit(&var, max_cbm_len, zero_bit) < max_cbm_len)
+		return false;
+
+	return true;
+}
+
+static int clos_cbm_table_read(u32 closid, unsigned long *l3_cbm)
+{
+	u32 maxid = boot_cpu_data.x86_cache_max_closid;
+
+	lockdep_assert_held(&rdt_group_mutex);
+
+	if (closid >= maxid)
+		return -EINVAL;
+
+	*l3_cbm = cctable[closid].l3_cbm;
+
+	return 0;
+}
+
+/*
+ * clos_cbm_table_update() - Update a clos cbm table entry.
+ * @closid: the closid whose cbm needs to be updated
+ * @cbm: the new cbm value that has to be updated
+ *
+ * This assumes the cbm is validated as per the interface requirements
+ * and the cache allocation requirements(through the cbm_validate).
+ */
+static int clos_cbm_table_update(u32 closid, unsigned long cbm)
+{
+	u32 maxid = boot_cpu_data.x86_cache_max_closid;
+
+	lockdep_assert_held(&rdt_group_mutex);
+
+	if (closid >= maxid)
+		return -EINVAL;
+
+	cctable[closid].l3_cbm = cbm;
+
+	return 0;
+}
+
+static bool cbm_search(unsigned long cbm, u32 *closid)
+{
+	u32 maxid = boot_cpu_data.x86_cache_max_closid;
+	u32 i;
+
+	for (i = 0; i < maxid; i++) {
+		if (cctable[i].clos_refcnt &&
+		    bitmap_equal(&cbm, &cctable[i].l3_cbm, MAX_CBM_LENGTH)) {
+			*closid = i;
+			return true;
+		}
+	}
+
+	return false;
+}
+
+static void closcbm_map_dump(void)
+{
+	u32 i;
+
+	pr_debug("CBMMAP\n");
+	for (i = 0; i < boot_cpu_data.x86_cache_max_closid; i++) {
+		pr_debug("l3_cbm: 0x%x,clos_refcnt: %u\n",
+		 (unsigned int)cctable[i].l3_cbm, cctable[i].clos_refcnt);
+	}
+}
+
+static void msr_cpu_update(void *arg)
+{
+	struct rdt_remote_data *info = arg;
+
+	wrmsrl(info->msr, info->val);
+}
+
+/*
+ * msr_update_all() - Update the msr for all packages.
+ */
+static inline void msr_update_all(int msr, u64 val)
+{
+	struct rdt_remote_data info;
+
+	info.msr = msr;
+	info.val = val;
+	on_each_cpu_mask(&rdt_cpumask, msr_cpu_update, &info, 1);
+}
+
+static inline bool rdt_cpumask_update(int cpu)
+{
+	cpumask_and(&tmp_cpumask, &rdt_cpumask, topology_core_cpumask(cpu));
+	if (cpumask_empty(&tmp_cpumask)) {
+		cpumask_set_cpu(cpu, &rdt_cpumask);
+		return true;
+	}
+
+	return false;
+}
+
 static int __init intel_rdt_late_init(void)
 {
 	struct cpuinfo_x86 *c = &boot_cpu_data;
 	u32 maxid, max_cbm_len;
-	int err = 0, size;
+	int err = 0, size, i;
 
 	if (!cpu_has(c, X86_FEATURE_CAT_L3))
 		return -ENODEV;
@@ -109,6 +238,8 @@ static int __init intel_rdt_late_init(void)
 		goto out_err;
 	}
 
+	for_each_online_cpu(i)
+		rdt_cpumask_update(i);
 	pr_info("Intel cache allocation enabled\n");
 out_err:
 
-- 
2.5.0

[toc] | [prev] | [next] | [standalone]

#1448778 — Re: [PATCH 04/32] x86/intel_rdt: Add L3 cache capacity bitmask management

On Tue, Jul 12, 2016 at 06:02:37PM -0700, Fenghua Yu wrote:
> From: Vikas Shivappa <vikas.shivappa@linux.intel.com>
> 
> This patch adds different APIs to manage the L3 cache capacity bitmask.
> The capacity bit mask(CBM) needs to have only contiguous bits set. The
> current implementation has a global CBM for each class of service id.
> There are APIs added to update the CBM via MSR write to IA32_L3_MASK_n
> on all packages. Other APIs are to read and write entries to the
> clos_cbm_table.
> 
> Signed-off-by: Vikas Shivappa <vikas.shivappa@linux.intel.com>
> Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
> Reviewed-by: Tony Luck <tony.luck@intel.com>
> ---
>  arch/x86/include/asm/intel_rdt.h |   4 ++
>  arch/x86/kernel/cpu/intel_rdt.c  | 133 ++++++++++++++++++++++++++++++++++++++-
>  2 files changed, 136 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/x86/include/asm/intel_rdt.h b/arch/x86/include/asm/intel_rdt.h
> index 88b7643..4f45dc8 100644
> --- a/arch/x86/include/asm/intel_rdt.h
> +++ b/arch/x86/include/asm/intel_rdt.h
> @@ -3,6 +3,10 @@
>  
>  #ifdef CONFIG_INTEL_RDT
>  
> +#define MAX_CBM_LENGTH			32
> +#define IA32_L3_CBM_BASE		0xc90
> +#define CBM_FROM_INDEX(x)		(IA32_L3_CBM_BASE + x)
> +
>  struct clos_cbm_table {
>  	unsigned long l3_cbm;
>  	unsigned int clos_refcnt;
> diff --git a/arch/x86/kernel/cpu/intel_rdt.c b/arch/x86/kernel/cpu/intel_rdt.c
> index d79213a..6ad5b48 100644
> --- a/arch/x86/kernel/cpu/intel_rdt.c
> +++ b/arch/x86/kernel/cpu/intel_rdt.c
> @@ -34,8 +34,22 @@ static struct clos_cbm_table *cctable;
>   * closid availability bit map.
>   */
>  unsigned long *closmap;
> +/*
> + * Mask of CPUs for writing CBM values. We only need one CPU per-socket.
> + */
> +static cpumask_t rdt_cpumask;
> +/*
> + * Temporary cpumask used during hot cpu notificaiton handling. The usage
> + * is serialized by hot cpu locks.
> + */
> +static cpumask_t tmp_cpumask;
>  static DEFINE_MUTEX(rdt_group_mutex);
>  
> +struct rdt_remote_data {
> +	int msr;
> +	u64 val;
> +};
> +
>  static inline void closid_get(u32 closid)
>  {
>  	struct clos_cbm_table *cct = &cctable[closid];
> @@ -82,11 +96,126 @@ static void closid_put(u32 closid)
>  		closid_free(closid);
>  }
>  
> +static bool cbm_validate(unsigned long var)
> +{
> +	u32 max_cbm_len = boot_cpu_data.x86_cache_max_cbm_len;
> +	unsigned long first_bit, zero_bit;
> +	u64 max_cbm;
> +
> +	if (bitmap_weight(&var, max_cbm_len) < 1)
> +		return false;
> +
> +	max_cbm = (1ULL << max_cbm_len) - 1;
> +	if (var & ~max_cbm)
> +		return false;
> +
> +	first_bit = find_first_bit(&var, max_cbm_len);
> +	zero_bit = find_next_zero_bit(&var, max_cbm_len, first_bit);
> +
> +	if (find_next_bit(&var, max_cbm_len, zero_bit) < max_cbm_len)
> +		return false;
> +
> +	return true;
> +}
> +
> +static int clos_cbm_table_read(u32 closid, unsigned long *l3_cbm)
> +{
> +	u32 maxid = boot_cpu_data.x86_cache_max_closid;
> +
> +	lockdep_assert_held(&rdt_group_mutex);
> +
> +	if (closid >= maxid)
> +		return -EINVAL;
> +
> +	*l3_cbm = cctable[closid].l3_cbm;
> +
> +	return 0;
> +}
> +
> +/*
> + * clos_cbm_table_update() - Update a clos cbm table entry.
> + * @closid: the closid whose cbm needs to be updated
> + * @cbm: the new cbm value that has to be updated
> + *
> + * This assumes the cbm is validated as per the interface requirements
> + * and the cache allocation requirements(through the cbm_validate).
> + */
> +static int clos_cbm_table_update(u32 closid, unsigned long cbm)
> +{
> +	u32 maxid = boot_cpu_data.x86_cache_max_closid;
> +
> +	lockdep_assert_held(&rdt_group_mutex);
> +
> +	if (closid >= maxid)
> +		return -EINVAL;
> +
> +	cctable[closid].l3_cbm = cbm;
> +
> +	return 0;
> +}
> +
> +static bool cbm_search(unsigned long cbm, u32 *closid)
> +{
> +	u32 maxid = boot_cpu_data.x86_cache_max_closid;
> +	u32 i;
> +
> +	for (i = 0; i < maxid; i++) {
> +		if (cctable[i].clos_refcnt &&
> +		    bitmap_equal(&cbm, &cctable[i].l3_cbm, MAX_CBM_LENGTH)) {
> +			*closid = i;
> +			return true;
> +		}
> +	}
> +
> +	return false;
> +}
> +
> +static void closcbm_map_dump(void)
> +{
> +	u32 i;
> +
> +	pr_debug("CBMMAP\n");
> +	for (i = 0; i < boot_cpu_data.x86_cache_max_closid; i++) {
> +		pr_debug("l3_cbm: 0x%x,clos_refcnt: %u\n",
> +		 (unsigned int)cctable[i].l3_cbm, cctable[i].clos_refcnt);
> +	}
> +}
> +
> +static void msr_cpu_update(void *arg)
> +{
> +	struct rdt_remote_data *info = arg;
> +
> +	wrmsrl(info->msr, info->val);
> +}
> +
> +/*
> + * msr_update_all() - Update the msr for all packages.
> + */
> +static inline void msr_update_all(int msr, u64 val)
> +{
> +	struct rdt_remote_data info;
> +
> +	info.msr = msr;
> +	info.val = val;
> +	on_each_cpu_mask(&rdt_cpumask, msr_cpu_update, &info, 1);
> +}

How does this patchset handle the following condition:

6) Create reservations in such a way that the sum is larger than
total amount of cache, and CPU pinning (example from Karen Noel):

VM-1 on socket-1 with 80% of reservation.
VM-2 on socket-2 with 80% of reservation.
VM-1 pinned to socket-1.
VM-2 pinned to socket-2.


> +
> +static inline bool rdt_cpumask_update(int cpu)
> +{
> +	cpumask_and(&tmp_cpumask, &rdt_cpumask, topology_core_cpumask(cpu));
> +	if (cpumask_empty(&tmp_cpumask)) {
> +		cpumask_set_cpu(cpu, &rdt_cpumask);
> +		return true;
> +	}
> +
> +	return false;
> +}
> +
>  static int __init intel_rdt_late_init(void)
>  {
>  	struct cpuinfo_x86 *c = &boot_cpu_data;
>  	u32 maxid, max_cbm_len;
> -	int err = 0, size;
> +	int err = 0, size, i;
>  
>  	if (!cpu_has(c, X86_FEATURE_CAT_L3))
>  		return -ENODEV;
> @@ -109,6 +238,8 @@ static int __init intel_rdt_late_init(void)
>  		goto out_err;
>  	}
>  
> +	for_each_online_cpu(i)
> +		rdt_cpumask_update(i);
>  	pr_info("Intel cache allocation enabled\n");
>  out_err:
>  
> -- 
> 2.5.0

[toc] | [prev] | [next] | [standalone]

#1448801 — Re: [PATCH 04/32] x86/intel_rdt: Add L3 cache capacity bitmask management

On Fri, Jul 22, 2016 at 04:12:04AM -0300, Marcelo Tosatti wrote:
> How does this patchset handle the following condition:
> 
> 6) Create reservations in such a way that the sum is larger than
> total amount of cache, and CPU pinning (example from Karen Noel):
> 
> VM-1 on socket-1 with 80% of reservation.
> VM-2 on socket-2 with 80% of reservation.
> VM-1 pinned to socket-1.
> VM-2 pinned to socket-2.

That's legal, but perhaps we need a description of
overlapping cache reservations.

Hardware tells you how finely you can divide the cache (and this
information is shown in /sys/fs/resctrl/info/l3/max_cbm_len to save
you from digging in CPUID leaves).  E.g. on Broadwell the value is
20, so you can control cache allocations in 5% slices.

A bitmask defines which slices you can use (and h/w has the restriction
that you must have contiguous '1' bits in any mask).  So you can pick
your 80% using 0x0ffff, 0x1fffe, 0x3fffc, 0x7fff8 or 0xffff0.

There is no requirement that masks be exclusive of each other. So
you might pick the two extremes: 0x0ffff and 0xffff0 for your two
VM's in this example. Each would be allowed to allocate up to 80%,
but with a big overlap in the middle. Each has 20% exclusive, but
there is a 60% range in the middle that they would compete for.

Is this specific case useful? Possibly not.  I think the more common
overlap cases might be between processes that you know have shared
code/data. Also the case where some rdtgroup has access to allocate
in the entire cache (mask 0xfffff on Broadwell) and some other rdtgroups
have limited cache allocation with less bits in the mask.

-Tony

[toc] | [prev] | [next] | [standalone]

#1450327 — Re: [PATCH 04/32] x86/intel_rdt: Add L3 cache capacity bitmask management

On Fri, Jul 22, 2016 at 02:43:23PM -0700, Luck, Tony wrote:
> On Fri, Jul 22, 2016 at 04:12:04AM -0300, Marcelo Tosatti wrote:
> > How does this patchset handle the following condition:
> > 
> > 6) Create reservations in such a way that the sum is larger than
> > total amount of cache, and CPU pinning (example from Karen Noel):
> > 
> > VM-1 on socket-1 with 80% of reservation.
> > VM-2 on socket-2 with 80% of reservation.
> > VM-1 pinned to socket-1.
> > VM-2 pinned to socket-2.
> 
> That's legal, but perhaps we need a description of
> overlapping cache reservations.
> 
> Hardware tells you how finely you can divide the cache (and this
> information is shown in /sys/fs/resctrl/info/l3/max_cbm_len to save
> you from digging in CPUID leaves).  E.g. on Broadwell the value is
> 20, so you can control cache allocations in 5% slices.
> 
> A bitmask defines which slices you can use (and h/w has the restriction
> that you must have contiguous '1' bits in any mask).  So you can pick
> your 80% using 0x0ffff, 0x1fffe, 0x3fffc, 0x7fff8 or 0xffff0.
> 
> There is no requirement that masks be exclusive of each other. So
> you might pick the two extremes: 0x0ffff and 0xffff0 for your two
> VM's in this example. Each would be allowed to allocate up to 80%,
> but with a big overlap in the middle. Each has 20% exclusive, but
> there is a 60% range in the middle that they would compete for.

This are different sockets, so there is no competing/sharing of L3 cache
here: the question is about whether the interface allows the
user to specify that 80/80 reservation without complaining:
because the VM's are pinned, they will never actually
share the same L3 cache.

(haven't finished reading the patchset to be certain).

> Is this specific case useful? Possibly not.  I think the more common
> overlap cases might be between processes that you know have shared
> code/data. Also the case where some rdtgroup has access to allocate
> in the entire cache (mask 0xfffff on Broadwell) and some other
> rdtgroups
> have limited cache allocation with less bits in the mask.
>
> -Tony

All you have to do is to build the bitmask for a given processor
from the union of the tasks which have been scheduled on that
processor.

[toc] | [prev] | [next] | [standalone]

#1450348 — Re: [PATCH 04/32] x86/intel_rdt: Add L3 cache capacity bitmask management

You must specify a mask for each L3 cache. So you can achieve your 80/80 split either with one rdtgroup that has an 80% mask on each of the sockets and using affinity to make one VM run only on CPUs on one socket and the second VM on the other. 

Or separate rdtgroups for each VM that give them the 80% when they are on their own socket and the spare 20% if the wander off to the other socket.

Sent from my iPhone

> On Jul 25, 2016, at 19:13, Marcelo Tosatti <mtosatti@redhat.com> wrote:
> 
>> On Fri, Jul 22, 2016 at 02:43:23PM -0700, Luck, Tony wrote:
>>> On Fri, Jul 22, 2016 at 04:12:04AM -0300, Marcelo Tosatti wrote:
>>> How does this patchset handle the following condition:
>>> 
>>> 6) Create reservations in such a way that the sum is larger than
>>> total amount of cache, and CPU pinning (example from Karen Noel):
>>> 
>>> VM-1 on socket-1 with 80% of reservation.
>>> VM-2 on socket-2 with 80% of reservation.
>>> VM-1 pinned to socket-1.
>>> VM-2 pinned to socket-2.
>> 
>> That's legal, but perhaps we need a description of
>> overlapping cache reservations.
>> 
>> Hardware tells you how finely you can divide the cache (and this
>> information is shown in /sys/fs/resctrl/info/l3/max_cbm_len to save
>> you from digging in CPUID leaves).  E.g. on Broadwell the value is
>> 20, so you can control cache allocations in 5% slices.
>> 
>> A bitmask defines which slices you can use (and h/w has the restriction
>> that you must have contiguous '1' bits in any mask).  So you can pick
>> your 80% using 0x0ffff, 0x1fffe, 0x3fffc, 0x7fff8 or 0xffff0.
>> 
>> There is no requirement that masks be exclusive of each other. So
>> you might pick the two extremes: 0x0ffff and 0xffff0 for your two
>> VM's in this example. Each would be allowed to allocate up to 80%,
>> but with a big overlap in the middle. Each has 20% exclusive, but
>> there is a 60% range in the middle that they would compete for.
> 
> This are different sockets, so there is no competing/sharing of L3 cache
> here: the question is about whether the interface allows the
> user to specify that 80/80 reservation without complaining:
> because the VM's are pinned, they will never actually
> share the same L3 cache.
> 
> (haven't finished reading the patchset to be certain).
> 
>> Is this specific case useful? Possibly not.  I think the more common
>> overlap cases might be between processes that you know have shared
>> code/data. Also the case where some rdtgroup has access to allocate
>> in the entire cache (mask 0xfffff on Broadwell) and some other
>> rdtgroups
>> have limited cache allocation with less bits in the mask.
>> 
>> -Tony
> 
> All you have to do is to build the bitmask for a given processor
> from the union of the tasks which have been scheduled on that
> processor.
> 
> 

[toc] | [prev] | [next] | [standalone]

#1450701 — Re: [PATCH 04/32] x86/intel_rdt: Add L3 cache capacity bitmask management

On Sat, 23 Jul 2016, Marcelo Tosatti wrote:

> On Fri, Jul 22, 2016 at 02:43:23PM -0700, Luck, Tony wrote:
>> On Fri, Jul 22, 2016 at 04:12:04AM -0300, Marcelo Tosatti wrote:
>>> How does this patchset handle the following condition:
>>>
>>> 6) Create reservations in such a way that the sum is larger than
>>> total amount of cache, and CPU pinning (example from Karen Noel):
>>>
>>> VM-1 on socket-1 with 80% of reservation.
>>> VM-2 on socket-2 with 80% of reservation.
>>> VM-1 pinned to socket-1.
>>> VM-2 pinned to socket-2.
>>
>> That's legal, but perhaps we need a description of
>> overlapping cache reservations.
>>
>> Hardware tells you how finely you can divide the cache (and this
>> information is shown in /sys/fs/resctrl/info/l3/max_cbm_len to save
>> you from digging in CPUID leaves).  E.g. on Broadwell the value is
>> 20, so you can control cache allocations in 5% slices.
>>
>> A bitmask defines which slices you can use (and h/w has the restriction
>> that you must have contiguous '1' bits in any mask).  So you can pick
>> your 80% using 0x0ffff, 0x1fffe, 0x3fffc, 0x7fff8 or 0xffff0.
>>
>> There is no requirement that masks be exclusive of each other. So
>> you might pick the two extremes: 0x0ffff and 0xffff0 for your two
>> VM's in this example. Each would be allowed to allocate up to 80%,
>> but with a big overlap in the middle. Each has 20% exclusive, but
>> there is a 60% range in the middle that they would compete for.
>
> This are different sockets, so there is no competing/sharing of L3 cache
> here: the question is about whether the interface allows the
> user to specify that 80/80 reservation without complaining:
> because the VM's are pinned, they will never actually
> share the same L3 cache.
>
> (haven't finished reading the patchset to be certain).

This series adds the per-socket support (See 23/32) - which will be folded into 
these which will should make it easier to read like Thomas suggested. 
The first 12 patches are the same as the old ones with the cgroup interface..
with the cgroup we were stuck with the 
atomicity issue to support per-socket because there was no way to define an 
interface to guarentee atomicity to the user when he wants an allocation of 
masks across different sockets

(basically he could end up in a situation where he could get a mask on one 
socket but not able to get it on an other as he ran out of closids, now 
the interface does it all at once - requiring the user to specify the masks 
for all sockets if you see the interface in later patches..) You could still do 
the above case you say with cgroup by co-mounting cpusets
and global masks but end up wasting closids.

Thanks,
Vikas

[toc] | [prev] | [next] | [standalone]

#1441801 — [PATCH 05/32] x86/intel_rdt: Implement scheduling support for Intel RDT

From: Vikas Shivappa <vikas.shivappa@linux.intel.com>

Adds support for IA32_PQR_ASSOC MSR writes during task scheduling. For
Cache Allocation, MSR write would let the task fill in the cache
'subset' represented by the task's capacity bit mask.

The high 32 bits in the per processor MSR IA32_PQR_ASSOC represents the
CLOSid. During context switch kernel implements this by writing the
CLOSid of the task belongs to the CPU's IA32_PQR_ASSOC MSR.

This patch also implements a common software cache for IA32_PQR_MSR
(RMID 0:9, CLOSId 32:63) to be used by both Cache monitoring (CMT) and
Cache allocation. CMT updates the RMID where as cache_alloc updates the
CLOSid in the software cache. During scheduling when the new RMID/CLOSid
value is different from the cached values, IA32_PQR_MSR is updated.
Since the measured rdmsr latency for IA32_PQR_MSR is very high (~250
 cycles) this software cache is necessary to avoid reading the MSR to
compare the current CLOSid value.

The following considerations are done for the PQR MSR write so that it
minimally impacts scheduler hot path:
 - This path does not exist on any non-intel platforms.
 - On Intel platforms, this would not exist by default unless INTEL_RDT
 is enabled.
 - remains a no-op when INTEL_RDT is enabled and intel SKU does not
 support the feature.
 - When feature is available and enabled, never does MSR write till the
 user manually starts using one of the capacity bit masks.
 - MSR write is only done when there is a task with different Closid is
 scheduled on the CPU. Typically if the task groups are bound to be
 scheduled on a set of CPUs, the number of MSR writes is greatly
 reduced.
 - A per CPU cache of CLOSids is maintained to do the check so that we
 don't have to do a rdmsr which actually costs a lot of cycles.

Signed-off-by: Vikas Shivappa <vikas.shivappa@linux.intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Reviewed-by: Tony Luck <tony.luck@intel.com>
---
 arch/x86/events/intel/cqm.c       | 26 +++-----------------------
 arch/x86/include/asm/intel_rdt.h  | 28 ++++++++++++++++++++++++++++
 arch/x86/include/asm/pqr_common.h | 27 +++++++++++++++++++++++++++
 arch/x86/kernel/cpu/intel_rdt.c   | 25 +++++++++++++++++++++++++
 arch/x86/kernel/process_64.c      |  6 ++++++
 5 files changed, 89 insertions(+), 23 deletions(-)
 create mode 100644 arch/x86/include/asm/pqr_common.h

diff --git a/arch/x86/events/intel/cqm.c b/arch/x86/events/intel/cqm.c
index 7b5fd81..ac76e43 100644
--- a/arch/x86/events/intel/cqm.c
+++ b/arch/x86/events/intel/cqm.c
@@ -7,9 +7,9 @@
 #include <linux/perf_event.h>
 #include <linux/slab.h>
 #include <asm/cpu_device_id.h>
+#include <asm/pqr_common.h>
 #include "../perf_event.h"
 
-#define MSR_IA32_PQR_ASSOC	0x0c8f
 #define MSR_IA32_QM_CTR		0x0c8e
 #define MSR_IA32_QM_EVTSEL	0x0c8d
 
@@ -24,32 +24,13 @@ static unsigned int cqm_l3_scale; /* supposedly cacheline size */
 static bool cqm_enabled, mbm_enabled;
 unsigned int mbm_socket_max;
 
-/**
- * struct intel_pqr_state - State cache for the PQR MSR
- * @rmid:		The cached Resource Monitoring ID
- * @closid:		The cached Class Of Service ID
- * @rmid_usecnt:	The usage counter for rmid
- *
- * The upper 32 bits of MSR_IA32_PQR_ASSOC contain closid and the
- * lower 10 bits rmid. The update to MSR_IA32_PQR_ASSOC always
- * contains both parts, so we need to cache them.
- *
- * The cache also helps to avoid pointless updates if the value does
- * not change.
- */
-struct intel_pqr_state {
-	u32			rmid;
-	u32			closid;
-	int			rmid_usecnt;
-};
-
 /*
  * The cached intel_pqr_state is strictly per CPU and can never be
  * updated from a remote CPU. Both functions which modify the state
  * (intel_cqm_event_start and intel_cqm_event_stop) are called with
  * interrupts disabled, which is sufficient for the protection.
  */
-static DEFINE_PER_CPU(struct intel_pqr_state, pqr_state);
+DEFINE_PER_CPU(struct intel_pqr_state, pqr_state);
 static struct hrtimer *mbm_timers;
 /**
  * struct sample - mbm event's (local or total) data
@@ -476,9 +457,9 @@ static void cqm_mask_call(struct rmid_read *rr)
  */
 static u32 intel_cqm_xchg_rmid(struct perf_event *group, u32 rmid)
 {
-	struct perf_event *event;
 	struct list_head *head = &group->hw.cqm_group_entry;
 	u32 old_rmid = group->hw.cqm_rmid;
+	struct perf_event *event;
 
 	lockdep_assert_held(&cache_mutex);
 
@@ -1583,7 +1564,6 @@ static void intel_cqm_cpu_starting(unsigned int cpu)
 	struct cpuinfo_x86 *c = &cpu_data(cpu);
 
 	state->rmid = 0;
-	state->closid = 0;
 	state->rmid_usecnt = 0;
 
 	WARN_ON(c->x86_cache_max_rmid != cqm_max_rmid);
diff --git a/arch/x86/include/asm/intel_rdt.h b/arch/x86/include/asm/intel_rdt.h
index 4f45dc8..afb6da3 100644
--- a/arch/x86/include/asm/intel_rdt.h
+++ b/arch/x86/include/asm/intel_rdt.h
@@ -3,14 +3,42 @@
 
 #ifdef CONFIG_INTEL_RDT
 
+#include <linux/jump_label.h>
+
 #define MAX_CBM_LENGTH			32
 #define IA32_L3_CBM_BASE		0xc90
 #define CBM_FROM_INDEX(x)		(IA32_L3_CBM_BASE + x)
 
+extern struct static_key rdt_enable_key;
+void __intel_rdt_sched_in(void *dummy);
+
 struct clos_cbm_table {
 	unsigned long l3_cbm;
 	unsigned int clos_refcnt;
 };
 
+/*
+ * intel_rdt_sched_in() - Writes the task's CLOSid to IA32_PQR_MSR
+ *
+ * Following considerations are made so that this has minimal impact
+ * on scheduler hot path:
+ * - This will stay as no-op unless we are running on an Intel SKU
+ * which supports L3 cache allocation.
+ * - Caches the per cpu CLOSid values and does the MSR write only
+ * when a task with a different CLOSid is scheduled in.
+ */
+static inline void intel_rdt_sched_in(void)
+{
+	/*
+	 * Call the schedule in code only when RDT is enabled.
+	 */
+	if (static_key_false(&rdt_enable_key))
+		__intel_rdt_sched_in(NULL);
+}
+
+#else
+
+static inline void intel_rdt_sched_in(void) {}
+
 #endif
 #endif
diff --git a/arch/x86/include/asm/pqr_common.h b/arch/x86/include/asm/pqr_common.h
new file mode 100644
index 0000000..11e985c
--- /dev/null
+++ b/arch/x86/include/asm/pqr_common.h
@@ -0,0 +1,27 @@
+#ifndef _X86_RDT_H_
+#define _X86_RDT_H_
+
+#define MSR_IA32_PQR_ASSOC	0x0c8f
+
+/**
+ * struct intel_pqr_state - State cache for the PQR MSR
+ * @rmid:		The cached Resource Monitoring ID
+ * @closid:		The cached Class Of Service ID
+ * @rmid_usecnt:	The usage counter for rmid
+ *
+ * The upper 32 bits of MSR_IA32_PQR_ASSOC contain closid and the
+ * lower 10 bits rmid. The update to MSR_IA32_PQR_ASSOC always
+ * contains both parts, so we need to cache them.
+ *
+ * The cache also helps to avoid pointless updates if the value does
+ * not change.
+ */
+struct intel_pqr_state {
+	u32			rmid;
+	u32			closid;
+	int			rmid_usecnt;
+};
+
+DECLARE_PER_CPU(struct intel_pqr_state, pqr_state);
+
+#endif
diff --git a/arch/x86/kernel/cpu/intel_rdt.c b/arch/x86/kernel/cpu/intel_rdt.c
index 6ad5b48..8379df8 100644
--- a/arch/x86/kernel/cpu/intel_rdt.c
+++ b/arch/x86/kernel/cpu/intel_rdt.c
@@ -24,6 +24,8 @@
 
 #include <linux/slab.h>
 #include <linux/err.h>
+#include <linux/sched.h>
+#include <asm/pqr_common.h>
 #include <asm/intel_rdt.h>
 
 /*
@@ -44,12 +46,33 @@ static cpumask_t rdt_cpumask;
  */
 static cpumask_t tmp_cpumask;
 static DEFINE_MUTEX(rdt_group_mutex);
+struct static_key __read_mostly rdt_enable_key = STATIC_KEY_INIT_FALSE;
 
 struct rdt_remote_data {
 	int msr;
 	u64 val;
 };
 
+void __intel_rdt_sched_in(void *dummy)
+{
+	struct intel_pqr_state *state = this_cpu_ptr(&pqr_state);
+	u32 closid = current->closid;
+
+	if (closid == state->closid)
+		return;
+
+	wrmsr(MSR_IA32_PQR_ASSOC, state->rmid, closid);
+	state->closid = closid;
+}
+
+/*
+ * Synchronize the IA32_PQR_ASSOC MSR of all currently running tasks.
+ */
+static inline void closid_tasks_sync(void)
+{
+	on_each_cpu_mask(cpu_online_mask, __intel_rdt_sched_in, NULL, 1);
+}
+
 static inline void closid_get(u32 closid)
 {
 	struct clos_cbm_table *cct = &cctable[closid];
@@ -240,6 +263,8 @@ static int __init intel_rdt_late_init(void)
 
 	for_each_online_cpu(i)
 		rdt_cpumask_update(i);
+
+	static_key_slow_inc(&rdt_enable_key);
 	pr_info("Intel cache allocation enabled\n");
 out_err:
 
diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c
index 6e789ca..71fc41a 100644
--- a/arch/x86/kernel/process_64.c
+++ b/arch/x86/kernel/process_64.c
@@ -48,6 +48,7 @@
 #include <asm/syscalls.h>
 #include <asm/debugreg.h>
 #include <asm/switch_to.h>
+#include <asm/intel_rdt.h>
 #include <asm/xen/hypervisor.h>
 
 asmlinkage extern void ret_from_fork(void);
@@ -472,6 +473,11 @@ __switch_to(struct task_struct *prev_p, struct task_struct *next_p)
 			loadsegment(ss, __KERNEL_DS);
 	}
 
+	/*
+	 * Load the Intel cache allocation PQR MSR.
+	 */
+	intel_rdt_sched_in();
+
 	return prev_p;
 }
 
-- 
2.5.0

[toc] | [prev] | [next] | [standalone]

#1449639 — Re: [PATCH 05/32] x86/intel_rdt: Implement scheduling support for Intel RDT

On 12 July 2016 at 20:02, Fenghua Yu <fenghua.yu@intel.com> wrote:
> diff --git a/arch/x86/include/asm/intel_rdt.h b/arch/x86/include/asm/intel_rdt.h
> index 4f45dc8..afb6da3 100644
> --- a/arch/x86/include/asm/intel_rdt.h
> +++ b/arch/x86/include/asm/intel_rdt.h
> @@ -3,14 +3,42 @@
>
>  #ifdef CONFIG_INTEL_RDT
>
> +#include <linux/jump_label.h>
> +
>  #define MAX_CBM_LENGTH                 32
>  #define IA32_L3_CBM_BASE               0xc90
>  #define CBM_FROM_INDEX(x)              (IA32_L3_CBM_BASE + x)
>
> +extern struct static_key rdt_enable_key;
> +void __intel_rdt_sched_in(void *dummy);
> +
>  struct clos_cbm_table {
>         unsigned long l3_cbm;
>         unsigned int clos_refcnt;
>  };
>
> +/*
> + * intel_rdt_sched_in() - Writes the task's CLOSid to IA32_PQR_MSR
> + *
> + * Following considerations are made so that this has minimal impact
> + * on scheduler hot path:
> + * - This will stay as no-op unless we are running on an Intel SKU
> + * which supports L3 cache allocation.
> + * - Caches the per cpu CLOSid values and does the MSR write only
> + * when a task with a different CLOSid is scheduled in.
> + */
> +static inline void intel_rdt_sched_in(void)
> +{
> +       /*
> +        * Call the schedule in code only when RDT is enabled.
> +        */
> +       if (static_key_false(&rdt_enable_key))
> +               __intel_rdt_sched_in(NULL);

static_key_false() is deprecated.  I think this should be
static_branch_unlikely().

> diff --git a/arch/x86/kernel/cpu/intel_rdt.c b/arch/x86/kernel/cpu/intel_rdt.c
> index 6ad5b48..8379df8 100644
> --- a/arch/x86/kernel/cpu/intel_rdt.c
> +++ b/arch/x86/kernel/cpu/intel_rdt.c
> @@ -44,12 +46,33 @@ static cpumask_t rdt_cpumask;
>   */
>  static cpumask_t tmp_cpumask;
>  static DEFINE_MUTEX(rdt_group_mutex);
> +struct static_key __read_mostly rdt_enable_key = STATIC_KEY_INIT_FALSE;
>

Similarly, this should be DEFINE_STATIC_KEY_FALSE(rdt_enable_key);


Thanks
Nilay

[toc] | [prev] | [next] | [standalone]

#1449646 — Re: [PATCH 05/32] x86/intel_rdt: Implement scheduling support for Intel RDT

On 12 July 2016 at 20:02, Fenghua Yu <fenghua.yu@intel.com> wrote:
> From: Vikas Shivappa <vikas.shivappa@linux.intel.com>
>
> Adds support for IA32_PQR_ASSOC MSR writes during task scheduling. For
> Cache Allocation, MSR write would let the task fill in the cache
> 'subset' represented by the task's capacity bit mask.
>
> The high 32 bits in the per processor MSR IA32_PQR_ASSOC represents the
> CLOSid. During context switch kernel implements this by writing the
> CLOSid of the task belongs to the CPU's IA32_PQR_ASSOC MSR.
>
> This patch also implements a common software cache for IA32_PQR_MSR
> (RMID 0:9, CLOSId 32:63) to be used by both Cache monitoring (CMT) and
> Cache allocation. CMT updates the RMID where as cache_alloc updates the
> CLOSid in the software cache. During scheduling when the new RMID/CLOSid
> value is different from the cached values, IA32_PQR_MSR is updated.
> Since the measured rdmsr latency for IA32_PQR_MSR is very high (~250
>  cycles) this software cache is necessary to avoid reading the MSR to
> compare the current CLOSid value.
>
> The following considerations are done for the PQR MSR write so that it
> minimally impacts scheduler hot path:
>  - This path does not exist on any non-intel platforms.
>  - On Intel platforms, this would not exist by default unless INTEL_RDT
>  is enabled.
>  - remains a no-op when INTEL_RDT is enabled and intel SKU does not
>  support the feature.
>  - When feature is available and enabled, never does MSR write till the
>  user manually starts using one of the capacity bit masks.
>  - MSR write is only done when there is a task with different Closid is
>  scheduled on the CPU. Typically if the task groups are bound to be
>  scheduled on a set of CPUs, the number of MSR writes is greatly
>  reduced.
>  - A per CPU cache of CLOSids is maintained to do the check so that we
>  don't have to do a rdmsr which actually costs a lot of cycles.
>

I was thinking more about this software caching of CLOSids.  How
likely do you think these CLOSids would be found cached?  I think the
software cache would be very infrequently accessed, so it seems you
are likely to miss these in all levels of cache hierarchy and more
likely to have to fetch these from the main memory, which itself might
cost ~250 cycles.

--
Nilay

[toc] | [prev] | [next] | [standalone]

#1449703 — Re: [PATCH 05/32] x86/intel_rdt: Implement scheduling support for Intel RDT

On Mon, Jul 25, 2016 at 11:31:24AM -0500, Nilay Vaish wrote:
> I was thinking more about this software caching of CLOSids.  How
> likely do you think these CLOSids would be found cached?  I think the
> software cache would be very infrequently accessed, so it seems you
> are likely to miss these in all levels of cache hierarchy and more
> likely to have to fetch these from the main memory, which itself might
> cost ~250 cycles.

We need to avoid reading the PQR_ASSOC MSR (which would cost far
more than 250 cycles).  Life is complicated here because this
MSR contains the CLOSID in the upper half, and the RMID (owned
by the perf code to measure cache occupancy and memory bandwidth)
in the lower half.

I've wondered whether it would be possible to include both the
CLOSID and the RMID in either the task or thread struct (in some
spot that was likely to be pulled into cache by access to adjacent
members during context switch).  This would smooth the path in
context switch (since we can look at the values from the previous
process and compare them to the ones for the new process to see
whether we need to do the wrmsr()).  But it likely needs a bunch
of other changes to update the values when a process migrates
between cpus to get the closid precedence that tglx requires.

-Tony

[toc] | [prev] | [next] | [standalone]

#1450237 — Re: [PATCH 05/32] x86/intel_rdt: Implement scheduling support for Intel RDT

On Mon, Jul 25, 2016 at 11:05 AM Luck, Tony <tony.luck@intel.com> wrote:
>
> On Mon, Jul 25, 2016 at 11:31:24AM -0500, Nilay Vaish wrote:
> > I was thinking more about this software caching of CLOSids.  How
> > likely do you think these CLOSids would be found cached?  I think the
> > software cache would be very infrequently accessed, so it seems you
> > are likely to miss these in all levels of cache hierarchy and more
> > likely to have to fetch these from the main memory, which itself might
> > cost ~250 cycles.
>
> We need to avoid reading the PQR_ASSOC MSR (which would cost far
> more than 250 cycles).  Life is complicated here because this
> MSR contains the CLOSID in the upper half, and the RMID (owned
> by the perf code to measure cache occupancy and memory bandwidth)
> in the lower half.


On my Haswell machine, writing PQR_ASSOC_MSR takes about 380 cycles.
As Tony said, CQM/CMT writes to the same register, and it does it
twice (once to delete the old event, once to add the new one).

So, if a CQM/CMT or MBM is used with CAT, there will be 3 writes to
PQR_ASSOC_MSR per context switch and it's quite likely that the
software cache's cache line will be there for the 2 last writes.

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | linux.kernel

csiph-web