Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1497607 > unrolled thread

[PATCH v3 00/18] Intel Cache Allocation Technology

Back to article view | Back to linux.kernel

Contents

  [PATCH v3 00/18] Intel Cache Allocation Technology "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-08 01:50 +0200
    [PATCH v3 04/18] x86/intel_rdt: Feature discovery "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-08 01:50 +0200
      Re: [PATCH v3 04/18] x86/intel_rdt: Feature discovery Nilay Vaish <nilayvaish@gmail.com> - 2016-10-08 19:20 +0200
        Re: [PATCH v3 04/18] x86/intel_rdt: Feature discovery Fenghua Yu <fenghua.yu@intel.com> - 2016-10-08 20:00 +0200
          Re: [PATCH v3 04/18] x86/intel_rdt: Feature discovery Borislav Petkov <bp@suse.de> - 2016-10-08 22:00 +0200
            Re: [PATCH v3 04/18] x86/intel_rdt: Feature discovery Nilay Vaish <nilayvaish@gmail.com> - 2016-10-11 19:10 +0200
              Re: [PATCH v3 04/18] x86/intel_rdt: Feature discovery Borislav Petkov <bp@suse.de> - 2016-10-11 19:10 +0200
        Re: [PATCH v3 04/18] x86/intel_rdt: Feature discovery Dave Hansen <dave.hansen@intel.com> - 2016-10-10 18:10 +0200
          Re: [PATCH v3 04/18] x86/intel_rdt: Feature discovery Borislav Petkov <bp@suse.de> - 2016-10-10 18:20 +0200
    [PATCH v3 10/18] x86/intel_rdt: Build structures for each resource based on cache topology "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-08 01:50 +0200
      Re: [PATCH v3 10/18] x86/intel_rdt: Build structures for each  resource based on cache topology Nilay Vaish <nilayvaish@gmail.com> - 2016-10-10 00:00 +0200
    [PATCH v3 16/18] x86/intel_rdt: Add schemata file "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-08 01:50 +0200
    [PATCH v3 07/18] x86/intel_rdt: Add Haswell feature discovery "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-08 01:50 +0200
      Re: [PATCH v3 07/18] x86/intel_rdt: Add Haswell feature discovery Borislav Petkov <bp@suse.de> - 2016-10-09 13:50 +0200
        Re: [PATCH v3 07/18] x86/intel_rdt: Add Haswell feature discovery Fenghua Yu <fenghua.yu@intel.com> - 2016-10-09 16:10 +0200
          Re: [PATCH v3 07/18] x86/intel_rdt: Add Haswell feature discovery Borislav Petkov <bp@suse.de> - 2016-10-09 18:30 +0200
            Re: [PATCH v3 07/18] x86/intel_rdt: Add Haswell feature discovery "Luck, Tony" <tony.luck@intel.com> - 2016-10-10 21:00 +0200
              Re: [PATCH v3 07/18] x86/intel_rdt: Add Haswell feature discovery Borislav Petkov <bp@suse.de> - 2016-10-11 13:20 +0200
                RE: [PATCH v3 07/18] x86/intel_rdt: Add Haswell feature discovery "Luck, Tony" <tony.luck@intel.com> - 2016-10-11 17:00 +0200
    [PATCH v3 13/18] x86/intel_rdt: Add mkdir to resctrl file system "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-08 01:50 +0200
      Re: [PATCH v3 13/18] x86/intel_rdt: Add mkdir to resctrl file system Nilay Vaish <nilayvaish@gmail.com> - 2016-10-10 20:00 +0200
    [RFC PATCH 19/18] x86/intel_rdt: Add support for L2 cache allocation "Luck, Tony" <tony.luck@intel.com> - 2016-10-08 02:00 +0200

Page 1 of 2  [1] 2  Next page →

#1497607 — [PATCH v3 00/18] Intel Cache Allocation Technology

From: Fenghua Yu <fenghua.yu@intel.com>

[I know we are in the middle of the merge window ... just putting
these out there now so that if anyone has a boring meeting or plane
ride they can have something interesting to do]

Dave:	Got rid of the atomic ops in fork() ... in fact there is no hook
	at all now. Membership in a resource group is indicated by the
	value of the current->closid field. So fork overhead is just
	the extra time to copy the task_struct that is one "int" bigger.
	We also got rid of some memory acceses in intel_rdt_sched_in()
	(which is also now fully in-line, so no function call). Maybe
	current->closid could be re-positioned inside the task_struct
	to get it into a warmer cache location.

Shaohua:Sorry, no "procs" file yet.  It can be added as another patch
	(hint: if you write it, it will happen faster :-)

PeterZ:	Using bitfields for the CPUID field extraction.

Nilay:	Using DEFINE_STATIC_KEY_FALSE now.

Boris:	Using init_scattered_cpuid_features()

Thomas:	So many comments ... I hope we got them all covered.

Totally revamped patch series to make review easier. Here's a
guide to what is in each patch, and what you can expect to see
on a running system with patches up to each point applied. Some
code got copied ... so Vikas deserves some Author credit, but I
didn't track which bits ended up in which new piece.

0001-Documentation-ABI-Add-a-document-entry-for-cache-id.patch
0002-cacheinfo-Introduce-cache-id.patch
0003-x86-intel_cacheinfo-Enable-cache-id-in-x86.patch

	These three define an "id" for each cache ... we need a "name"
	for a cache so we can say what restrictions to apply to each
	cache in the system.  All you will see at this point is an
	extra "id" file in each /sys/devices/system/cpu/cpu*/cache/index*/
	directory.

0004-x86-intel_rdt-Feature-discovery.patch

	Look at CPUID for the features related to cache allocation.
	At this point /proc/cpuinfo shows extra flags for the features
	found on your system.

0005-Documentation-x86-Documentation-for-Intel-resource-a.patch

	Documentation patch could be anywhere in this sequence. We
	put in early so you can read it to see how to use the
	interface.

0006-x86-intel_rdt-Add-CONFIG-Makefile-and-basic-initiali.patch

	Add CONFIG_INTEL_RDT (default "n" ... you'll have to set
	it to have this, and all the following patches do anything).
	Template driver here just checks for features and spams
	the console with one line for each.

0007-x86-intel_rdt-Add-Haswell-feature-discovery.patch

	There are some Haswell systems that support cache allocation,
	but they were made before the CPUID bits were fully defined.
	So we check by probing the PQR_ASSOC MSR to see if CLOSID
	bits stick. Unless you have one of these Haswells, you won't
	see any difference here.

0008-x86-intel_rdt-Pick-up-L3-RDT-parameters-from-CPUID.patch

	This is all new code, not seen in the previous versions of this
	patch series. L3 cache allocation is just the first of several
	resource control features. Define rdt_resource structure that
	contains all the useful things we need to know about a resource.
	Pick up the parameters for the resource from CPUID. The console
	spam strings change format here.

0009-x86-cqm-Move-PQR_ASSOC-management-code-into-generic-.patch

	The PQR_ASSOC MSR has a field for the CLOSID (which we need
	define which allocation rules are in effect). But it also
	contains the RMID (used by CQM and MBM perf monitoring).
	The perf code got here first, but defined structures that
	make it easy for the two systems to co-exist without stomping
	on each other. This patch moves the relevant parts into a
	common header file and changes the scope from "static" to
	global so we can access them. No visible change.

0010-x86-intel_rdt-Build-structures-for-each-resource-bas.patch

	For each enabled resource, we build a list of "rdt_domains" based
	on hotplug cpu notifications. Since we only have L3 at this point,
	this is just a list of L3 caches (named by the "id" established
	in the first three patches). As each cache is found we initialize
	the array of CBMs (cache bit masks). No visible change here.

0011-x86-intel_rdt-Add-basic-resctrl-filesystem-support.patch

	Our interface is a kernfs backed file system. Establish the
	mount point, and provide mount/unmount functionality.
	At this point "/sys/fs/resctrl" appears. You can mount and
	unmount the resctrl file system (if your system supports
	code/data prioritization, you can use the "cdp" mount option).
	The file system is empty and doesn't allow creation of any
	files or subdirectories.

0012-x86-intel_rdt-Add-info-files-to-resctrl-file-system.patch

	Parameters for each resource are buried in CPUID leaf 0x10.
	This isn't very user friendly for scripts and applications
	that want to configure resource allocation. Create an
	"info" directory, with a subdirectory for each resource
	containing a couple of useful parameters. Visible change:
	$ ls -l /sys/fs/resctrl/info/L3
	total 0
	-r--r--r-- 1 root root 0 Oct  7 11:20 cbm_val
	-r--r--r-- 1 root root 0 Oct  7 11:20 num_closid

0013-x86-intel_rdt-Add-mkdir-to-resctrl-file-system.patch

	Each resource group is represented by a directory in the
	resctrl file system. The root directory is the default group.
	Use "mkdir" to create new groups and "rmdir" to remove them.
	The maximum number of groups is defined by the effective
	number of CLOSIDs.
	Visible change: If you have CDP (and enable with the "cdp"
	mount option) you will find that you can only create half
	as many groups as without (e.g. 8 vs. 16 on Broadwell, but
	the default group uses one ... so actually 7, 15).

0014-x86-intel_rdt-Add-cpus-file.patch

	One of the control mechanisms for a resource group is the
	logical CPU. Initially all CPUs are assigned to the default
	group. They can be reassigned to other groups by writing
	a cpumask to the "cpus" file. See the documentation for what
	this means.
	Visible change: "cpus" file in the root, and automatically
	in each created subdirectory. You can "echo" masks to these
	files and watch as CPUs added to one group are removed from
	whatever group they previously belonged to. Removing a directory
	will give all CPUs owned by it back to the default (root)
	group.

0015-x86-intel_rdt-Add-tasks-files.patch

	Tasks can be assigned to resource groups by writing their PID
	to a "tasks" file (which removes the task from its previous
	group). Forked/cloned tasks inherit the group from their
	parent. You cannot remove a group (directory) that has any
	tasks assigned.
	Visible change: "tasks" files appear. E.g. (we see two tasks
	in the group, our shell, and the "cat" that it spawned).
	# echo $$ > p0/tasks; cat p0/tasks
	268890
	268914



0016-x86-intel_rdt-Add-schemata-file.patch

	The "schemata" file in each group/directory defines what
	access tasks controlled by this resource are permitted.
	One line per resource type. Fields for each instance of
	the resource. You redefine the access by wrting to the
	file in the same format.
	Visible change: "schemata" file which starts out with maximum
	allowed resources. E.g.
	$ cat schemata
	L3:0=fffff;1=fffff
	Now restrict this group to just 20% of L3 on first cache, but
	allow 50% on the second
	# echo L3:0=f;1=3ff > schemata

0017-x86-intel_rdt-Add-scheduler-hook.patch

	When context switching we check if we are changing resource
	groups for the new process, and update the PQR_ASSOC MSR with
	the new CLOSID if needed.
	Visble change: Everything should be working now. Tasks run with
	the permitted access to L3 cache.

0018-MAINTAINERS-Add-maintainer-for-Intel-RDT-resource-al.patch

	New files ... need a maintainer. Fenghua has the job.

Test note: The patches haven't been tested on 32-bit yet.


Fenghua Yu (15):
  Documentation, ABI: Add a document entry for cache id
  cacheinfo: Introduce cache id
  x86, intel_cacheinfo: Enable cache id in x86
  x86/intel_rdt: Feature discovery
  Documentation, x86: Documentation for Intel resource allocation user
    interface
  x86/intel_rdt: Add CONFIG, Makefile, and basic initialization
  x86/intel_rdt: Add Haswell feature discovery
  x86/intel_rdt: Pick up L3 RDT parameters from CPUID
  x86/cqm: Move PQR_ASSOC management code into generic code used by both
    CQM and CAT
  x86/intel_rdt: Add basic resctrl filesystem support
  x86/intel_rdt: Add "info" files to resctrl file system
  x86/intel_rdt: Add mkdir to resctrl file system
  x86/intel_rdt: Add tasks files
  x86/intel_rdt: Add scheduler hook
  MAINTAINERS: Add maintainer for Intel RDT resource allocation

Tony Luck (3):
  x86/intel_rdt: Build structures for each resource based on cache
    topology
  x86/intel_rdt: Add cpus file
  x86/intel_rdt: Add schemata file

 Documentation/ABI/testing/sysfs-devices-system-cpu |  16 +
 Documentation/x86/intel_rdt_ui.txt                 | 162 ++++
 MAINTAINERS                                        |   8 +
 arch/x86/Kconfig                                   |  12 +
 arch/x86/events/intel/cqm.c                        |  23 +-
 arch/x86/include/asm/cpufeatures.h                 |   5 +
 arch/x86/include/asm/intel_rdt.h                   | 205 +++++
 arch/x86/include/asm/intel_rdt_common.h            |  27 +
 arch/x86/kernel/cpu/Makefile                       |   2 +
 arch/x86/kernel/cpu/intel_cacheinfo.c              |  20 +
 arch/x86/kernel/cpu/intel_rdt.c                    | 294 +++++++
 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c           | 939 +++++++++++++++++++++
 arch/x86/kernel/cpu/intel_rdt_schemata.c           | 260 ++++++
 arch/x86/kernel/cpu/scattered.c                    |   3 +
 arch/x86/kernel/process_32.c                       |   4 +
 arch/x86/kernel/process_64.c                       |   4 +
 drivers/base/cacheinfo.c                           |   5 +
 include/linux/cacheinfo.h                          |   3 +
 include/linux/sched.h                              |   3 +
 include/uapi/linux/magic.h                         |   1 +
 20 files changed, 1975 insertions(+), 21 deletions(-)
 create mode 100644 Documentation/x86/intel_rdt_ui.txt
 create mode 100644 arch/x86/include/asm/intel_rdt.h
 create mode 100644 arch/x86/include/asm/intel_rdt_common.h
 create mode 100644 arch/x86/kernel/cpu/intel_rdt.c
 create mode 100644 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
 create mode 100644 arch/x86/kernel/cpu/intel_rdt_schemata.c

-- 
2.5.0

[toc] | [next] | [standalone]

#1497608 — [PATCH v3 04/18] x86/intel_rdt: Feature discovery

From: Fenghua Yu <fenghua.yu@intel.com>

Check CPUID leaves for all the Resource Director Technology (RDT)
Cache Allocation Technology (CAT) bits.

Prescence of allocation features:
  CPUID.(EAX=7H, ECX=0):EBX[bit 15]	X86_FEATURE_RDT_A

L2 and L3 caches are each separately enabled:
  CPUID.(EAX=10H, ECX=0):EBX[bit 1]	X86_FEATURE_CAT_L3
  CPUID.(EAX=10H, ECX=0):EBX[bit 2]	X86_FEATURE_CAT_L2

L3 cache may support independent control of allocation for
code and data (CDP = Code/Data Prioritization):
  CPUID.(EAX=10H, ECX=1):ECX[bit 2]	X86_FEATURE_CDP_L3

Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Signed-off-by: Tony Luck <tony.luck@intel.com>
---
 arch/x86/include/asm/cpufeatures.h | 5 +++++
 arch/x86/kernel/cpu/scattered.c    | 3 +++
 2 files changed, 8 insertions(+)

diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index 92a8308..64dd8274 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -196,6 +196,10 @@
 
 #define X86_FEATURE_INTEL_PT	( 7*32+15) /* Intel Processor Trace */
 
+#define X86_FEATURE_CAT_L3	( 7*32+16) /* Cache Allocation Technology L3 */
+#define X86_FEATURE_CAT_L2	( 7*32+17) /* Cache Allocation Technology L2 */
+#define X86_FEATURE_CDP_L3	( 7*32+18) /* Code and Data Prioritization L3 */
+
 /* Virtualization flags: Linux defined, word 8 */
 #define X86_FEATURE_TPR_SHADOW  ( 8*32+ 0) /* Intel TPR Shadow */
 #define X86_FEATURE_VNMI        ( 8*32+ 1) /* Intel Virtual NMI */
@@ -220,6 +224,7 @@
 #define X86_FEATURE_RTM		( 9*32+11) /* Restricted Transactional Memory */
 #define X86_FEATURE_CQM		( 9*32+12) /* Cache QoS Monitoring */
 #define X86_FEATURE_MPX		( 9*32+14) /* Memory Protection Extension */
+#define X86_FEATURE_RDT_A	( 9*32+15) /* Resource Director Technology Allocation */
 #define X86_FEATURE_AVX512F	( 9*32+16) /* AVX-512 Foundation */
 #define X86_FEATURE_AVX512DQ	( 9*32+17) /* AVX-512 DQ (Double/Quad granular) Instructions */
 #define X86_FEATURE_RDSEED	( 9*32+18) /* The RDSEED instruction */
diff --git a/arch/x86/kernel/cpu/scattered.c b/arch/x86/kernel/cpu/scattered.c
index 8cb57df..11f39a2 100644
--- a/arch/x86/kernel/cpu/scattered.c
+++ b/arch/x86/kernel/cpu/scattered.c
@@ -34,6 +34,9 @@ void init_scattered_cpuid_features(struct cpuinfo_x86 *c)
 		{ X86_FEATURE_INTEL_PT,		CR_EBX,25, 0x00000007, 0 },
 		{ X86_FEATURE_APERFMPERF,	CR_ECX, 0, 0x00000006, 0 },
 		{ X86_FEATURE_EPB,		CR_ECX, 3, 0x00000006, 0 },
+		{ X86_FEATURE_CAT_L3,		CR_EBX, 1, 0x00000010, 0 },
+		{ X86_FEATURE_CAT_L2,		CR_EBX, 2, 0x00000010, 0 },
+		{ X86_FEATURE_CDP_L3,		CR_ECX, 2, 0x00000010, 1 },
 		{ X86_FEATURE_HW_PSTATE,	CR_EDX, 7, 0x80000007, 0 },
 		{ X86_FEATURE_CPB,		CR_EDX, 9, 0x80000007, 0 },
 		{ X86_FEATURE_PROC_FEEDBACK,	CR_EDX,11, 0x80000007, 0 },
-- 
2.5.0

[toc] | [prev] | [next] | [standalone]

#1497789 — Re: [PATCH v3 04/18] x86/intel_rdt: Feature discovery

On 7 October 2016 at 21:45, Fenghua Yu <fenghua.yu@intel.com> wrote:
> From: Fenghua Yu <fenghua.yu@intel.com>
>
> Check CPUID leaves for all the Resource Director Technology (RDT)
> Cache Allocation Technology (CAT) bits.
>
> Prescence of allocation features:

Presence

> diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
> index 92a8308..64dd8274 100644
> --- a/arch/x86/include/asm/cpufeatures.h
> +++ b/arch/x86/include/asm/cpufeatures.h
> @@ -196,6 +196,10 @@
>
>  #define X86_FEATURE_INTEL_PT   ( 7*32+15) /* Intel Processor Trace */
>
> +#define X86_FEATURE_CAT_L3     ( 7*32+16) /* Cache Allocation Technology L3 */
> +#define X86_FEATURE_CAT_L2     ( 7*32+17) /* Cache Allocation Technology L2 */
> +#define X86_FEATURE_CDP_L3     ( 7*32+18) /* Code and Data Prioritization L3 */
> +
>  /* Virtualization flags: Linux defined, word 8 */
>  #define X86_FEATURE_TPR_SHADOW  ( 8*32+ 0) /* Intel TPR Shadow */
>  #define X86_FEATURE_VNMI        ( 8*32+ 1) /* Intel Virtual NMI */
> @@ -220,6 +224,7 @@
>  #define X86_FEATURE_RTM                ( 9*32+11) /* Restricted Transactional Memory */
>  #define X86_FEATURE_CQM                ( 9*32+12) /* Cache QoS Monitoring */
>  #define X86_FEATURE_MPX                ( 9*32+14) /* Memory Protection Extension */
> +#define X86_FEATURE_RDT_A      ( 9*32+15) /* Resource Director Technology Allocation */
>  #define X86_FEATURE_AVX512F    ( 9*32+16) /* AVX-512 Foundation */
>  #define X86_FEATURE_AVX512DQ   ( 9*32+17) /* AVX-512 DQ (Double/Quad granular) Instructions */
>  #define X86_FEATURE_RDSEED     ( 9*32+18) /* The RDSEED instruction */

I think these #defines are specific to Intel.  I would prefer if we
have _INTEL_ somewhere in them.

--
Nilay

[toc] | [prev] | [next] | [standalone]

#1497800 — Re: [PATCH v3 04/18] x86/intel_rdt: Feature discovery

On Sat, Oct 08, 2016 at 12:11:22PM -0500, Nilay Vaish wrote:
> On 7 October 2016 at 21:45, Fenghua Yu <fenghua.yu@intel.com> wrote:
> > From: Fenghua Yu <fenghua.yu@intel.com>
> >
> > Check CPUID leaves for all the Resource Director Technology (RDT)
> > Cache Allocation Technology (CAT) bits.
> >
> > Prescence of allocation features:
> 
> Presence
> 
> > diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
> > index 92a8308..64dd8274 100644
> > --- a/arch/x86/include/asm/cpufeatures.h
> > +++ b/arch/x86/include/asm/cpufeatures.h
> > @@ -196,6 +196,10 @@
> >
> >  #define X86_FEATURE_INTEL_PT   ( 7*32+15) /* Intel Processor Trace */
> >
> > +#define X86_FEATURE_CAT_L3     ( 7*32+16) /* Cache Allocation Technology L3 */
> > +#define X86_FEATURE_CAT_L2     ( 7*32+17) /* Cache Allocation Technology L2 */
> > +#define X86_FEATURE_CDP_L3     ( 7*32+18) /* Code and Data Prioritization L3 */
> > +
> >  /* Virtualization flags: Linux defined, word 8 */
> >  #define X86_FEATURE_TPR_SHADOW  ( 8*32+ 0) /* Intel TPR Shadow */
> >  #define X86_FEATURE_VNMI        ( 8*32+ 1) /* Intel Virtual NMI */
> > @@ -220,6 +224,7 @@
> >  #define X86_FEATURE_RTM                ( 9*32+11) /* Restricted Transactional Memory */
> >  #define X86_FEATURE_CQM                ( 9*32+12) /* Cache QoS Monitoring */
> >  #define X86_FEATURE_MPX                ( 9*32+14) /* Memory Protection Extension */
> > +#define X86_FEATURE_RDT_A      ( 9*32+15) /* Resource Director Technology Allocation */
> >  #define X86_FEATURE_AVX512F    ( 9*32+16) /* AVX-512 Foundation */
> >  #define X86_FEATURE_AVX512DQ   ( 9*32+17) /* AVX-512 DQ (Double/Quad granular) Instructions */
> >  #define X86_FEATURE_RDSEED     ( 9*32+18) /* The RDSEED instruction */
> 
> I think these #defines are specific to Intel.  I would prefer if we
> have _INTEL_ somewhere in them.

Is adding "Intel" in comment good?

+#define X86_FEATURE_CAT_L3     ( 7*32+16) /* Intel Cache Allocation Technology L3 */
+#define X86_FEATURE_CAT_L2     ( 7*32+17) /* Intel Cache Allocation Technology L2 */
+#define X86_FEATURE_CDP_L3     ( 7*32+18) /* Intel Code and Data Prioritization L3 */

Thanks.

-Fenghua

[toc] | [prev] | [next] | [standalone]

#1497812 — Re: [PATCH v3 04/18] x86/intel_rdt: Feature discovery

On Sat, Oct 08, 2016 at 01:54:54PM -0700, Fenghua Yu wrote:
> > I think these #defines are specific to Intel.  I would prefer if we
> > have _INTEL_ somewhere in them.

We don't generally add vendor names to those defines. Even more so if
the 0x0... leaf range is Intel-specific anyway.

> Is adding "Intel" in comment good?

I don't see any need if the leaf has already this heading:

/* Intel-defined CPU features, CPUID level 0x00000007:0 (ebx), word 9 */

-- 
Regards/Gruss,
    Boris.

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
-- 

[toc] | [prev] | [next] | [standalone]

#1499023 — Re: [PATCH v3 04/18] x86/intel_rdt: Feature discovery

On 8 October 2016 at 14:52, Borislav Petkov <bp@suse.de> wrote:
> On Sat, Oct 08, 2016 at 01:54:54PM -0700, Fenghua Yu wrote:
>> > I think these #defines are specific to Intel.  I would prefer if we
>> > have _INTEL_ somewhere in them.
>
> We don't generally add vendor names to those defines. Even more so if
> the 0x0... leaf range is Intel-specific anyway.
>
>> Is adding "Intel" in comment good?
>
> I don't see any need if the leaf has already this heading:
>
> /* Intel-defined CPU features, CPUID level 0x00000007:0 (ebx), word 9 */
>

I think we should go with Fenghua' suggestion on this.  Reading the
code around the edits from this patch, it seems word 7 is not owned by
anyone.  Both AMD and Intel seem to be using it.

--
Nilay

[toc] | [prev] | [next] | [standalone]

#1499024 — Re: [PATCH v3 04/18] x86/intel_rdt: Feature discovery

On Tue, Oct 11, 2016 at 11:57:52AM -0500, Nilay Vaish wrote:
> I think we should go with Fenghua' suggestion on this.  Reading the
> code around the edits from this patch, it seems word 7 is not owned by
> anyone.  Both AMD and Intel seem to be using it.

No, this is wrong. The 0x0 CPUID leaf is owned by Intel. Let me quote my
reply to Dave from yesterday:

There's no reason to do anything really: the 0x0... CPUID leaf is owned
by Intel and 0x8... by AMD. So far, if AMD implements a feature in the
Intel leaf, it sets the respective CPUID bit. So a feature can be
present in multiple vendors. Thus you don't need the vendor name in the
define."

-- 
Regards/Gruss,
    Boris.

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
-- 

[toc] | [prev] | [next] | [standalone]

#1498399 — Re: [PATCH v3 04/18] x86/intel_rdt: Feature discovery

On 10/08/2016 10:11 AM, Nilay Vaish wrote:
>> >  #define X86_FEATURE_RTM                ( 9*32+11) /* Restricted Transactional Memory */
>> >  #define X86_FEATURE_CQM                ( 9*32+12) /* Cache QoS Monitoring */
>> >  #define X86_FEATURE_MPX                ( 9*32+14) /* Memory Protection Extension */
>> > +#define X86_FEATURE_RDT_A      ( 9*32+15) /* Resource Director Technology Allocation */
>> >  #define X86_FEATURE_AVX512F    ( 9*32+16) /* AVX-512 Foundation */
>> >  #define X86_FEATURE_AVX512DQ   ( 9*32+17) /* AVX-512 DQ (Double/Quad granular) Instructions */
>> >  #define X86_FEATURE_RDSEED     ( 9*32+18) /* The RDSEED instruction */
> I think these #defines are specific to Intel.  I would prefer if we
> have _INTEL_ somewhere in them.

While that isn't a horrible idea, it's also not something that we've
enforced at *all* in the past.  Would you suggest that we do this only
for features in the generic CPUID leaves, or all features?

[toc] | [prev] | [next] | [standalone]

#1498405 — Re: [PATCH v3 04/18] x86/intel_rdt: Feature discovery

On Mon, Oct 10, 2016 at 09:01:44AM -0700, Dave Hansen wrote:
> While that isn't a horrible idea, it's also not something that we've
> enforced at *all* in the past.  Would you suggest that we do this only
> for features in the generic CPUID leaves, or all features?

There's no reason to do anything really: the 0x0... CPUID leaf is owned
by Intel and 0x8... by AMD. So far, if AMD implements a feature in the
Intel leaf, it sets the respective CPUID bit. So a feature can be
present in multiple vendors. Thus you don't need the vendor name in the
define.

-- 
Regards/Gruss,
    Boris.

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
-- 

[toc] | [prev] | [next] | [standalone]

#1497609 — [PATCH v3 10/18] x86/intel_rdt: Build structures for each resource based on cache topology

From: Tony Luck <tony.luck@intel.com>

We use the cpu hotplug notifier to catch each cpu in turn and look at
its cache topology w.r.t each of the resource groups. As we discover
new resources, we initialize the bitmask array for each to the default
(full access) value.

Signed-off-by: Tony Luck <tony.luck@intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
---
 arch/x86/include/asm/intel_rdt.h |  35 ++++++++++
 arch/x86/kernel/cpu/intel_rdt.c  | 135 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 170 insertions(+)

diff --git a/arch/x86/include/asm/intel_rdt.h b/arch/x86/include/asm/intel_rdt.h
index 251ac2a..bad8dc7 100644
--- a/arch/x86/include/asm/intel_rdt.h
+++ b/arch/x86/include/asm/intel_rdt.h
@@ -40,6 +40,39 @@ struct rdt_resource {
 		if (r->enabled)
 
 #define IA32_L3_CBM_BASE	0xc90
+
+/**
+ * struct rdt_domain - group of cpus sharing an RDT resource
+ * @list:	all instances of this resource
+ * @id:		unique id for this instance
+ * @cpu_mask:	which cpus share this resource
+ * @cbm:	array of cache bit masks (indexed by CLOSID)
+ */
+struct rdt_domain {
+	struct list_head	list;
+	int			id;
+	struct cpumask		cpu_mask;
+	u32			*cbm;
+};
+
+/**
+ * struct msr_param - set a range of MSRs from a domain
+ * @res:       The resource to use
+ * @low:       Beginning index from base MSR
+ * @high:      End index
+ */
+struct msr_param {
+	struct rdt_resource	*res;
+	int			low;
+	int			high;
+};
+
+extern struct static_key_false rdt_enable_key;
+extern struct mutex rdtgroup_mutex;
+
+int __init rdtgroup_init(void);
+
+extern struct rdtgroup *rdtgroup_default;
 extern struct rdt_resource rdt_resources_all[];
 
 enum {
@@ -64,4 +97,6 @@ union cpuid_0x10_1_edx {
 	} split;
 	unsigned int full;
 };
+
+void rdt_cbm_update(void *arg);
 #endif /* _ASM_X86_INTEL_RDT_H */
diff --git a/arch/x86/kernel/cpu/intel_rdt.c b/arch/x86/kernel/cpu/intel_rdt.c
index b9b1e9e..76b7476 100644
--- a/arch/x86/kernel/cpu/intel_rdt.c
+++ b/arch/x86/kernel/cpu/intel_rdt.c
@@ -24,12 +24,18 @@
 
 #define pr_fmt(fmt)	KBUILD_MODNAME ": " fmt
 
+#include <linux/cacheinfo.h>
 #include <linux/slab.h>
 #include <linux/err.h>
+#include <linux/cpuhotplug.h>
+
 #include <asm/intel_rdt_common.h>
 #include <asm/intel-family.h>
 #include <asm/intel_rdt.h>
 
+/* Mutex to protect rdtgroup access. */
+DEFINE_MUTEX(rdtgroup_mutex);
+
 int rdt_max_closid;
 
 #define domain_init(name) LIST_HEAD_INIT(rdt_resources_all[name].domains)
@@ -119,14 +125,143 @@ static inline bool get_rdt_resources(struct cpuinfo_x86 *c)
 	return ret;
 }
 
+static int get_cache_id(int cpu, int level)
+{
+	struct cpu_cacheinfo *ci = get_cpu_cacheinfo(cpu);
+	int i;
+
+	for (i = 0; i < ci->num_leaves; i++)
+		if (ci->info_list[i].level == level)
+			return ci->info_list[i].id;
+	return -1;
+}
+
+void rdt_cbm_update(void *arg)
+{
+	struct msr_param *m = (struct msr_param *)arg;
+	struct rdt_resource *r = m->res;
+	struct rdt_domain *d;
+	struct list_head *l;
+	int i, cpu = smp_processor_id();
+
+	list_for_each(l, &r->domains) {
+		d = list_entry(l, struct rdt_domain, list);
+		if (cpumask_test_cpu(cpu, &d->cpu_mask))
+			goto found;
+	}
+	pr_info_once("cpu %d not found in any domain for resource %s\n",
+		     cpu, r->name);
+
+found:
+	for (i = m->low; i < m->high; i++)
+		wrmsrl(r->msr_base + i, d->cbm[i]);
+}
+
+static void update_domain(int cpu, struct rdt_resource *r, int add)
+{
+	struct list_head *l;
+	struct rdt_domain *d;
+	int i, cache_id;
+
+	cache_id = get_cache_id(cpu, r->cache_level);
+
+	if (cache_id == -1) {
+		pr_info_once("Could't find cache id for cpu %d\n", cpu);
+		return;
+	}
+	list_for_each(l, &r->domains) {
+		d = list_entry(l, struct rdt_domain, list);
+		if (cache_id == d->id)
+			goto found;
+		if (cache_id < d->id)
+			break;
+	}
+	if (!add) {
+		pr_info_once("removed unknown cpu %d\n", cpu);
+		return;
+	}
+	d = kzalloc(sizeof(*d), GFP_KERNEL);
+	if (!d)
+		return;
+
+	d->id = cache_id;
+	d->cbm = kmalloc_array(r->max_closid, sizeof(*d->cbm), GFP_KERNEL);
+	if (!d->cbm) {
+		pr_info("Failed to alloc CBM array for cpu %d\n", cpu);
+		kfree(d);
+		return;
+	}
+	cpumask_set_cpu(cpu, &d->cpu_mask);
+	for (i = 0; i < r->max_closid; i++) {
+		d->cbm[i] = r->max_cbm;
+		wrmsrl(r->msr_base + i, d->cbm[i]);
+	}
+	list_add_tail(&d->list, l);
+	r->num_domains++;
+	return;
+
+found:
+	if (add) {
+		cpumask_set_cpu(cpu, &d->cpu_mask);
+	} else {
+		cpumask_clear_cpu(cpu, &d->cpu_mask);
+		if (cpumask_empty(&d->cpu_mask)) {
+			r->num_domains--;
+			kfree(d->cbm);
+			list_del(&d->list);
+			kfree(d);
+		}
+	}
+}
+
+static void rdt_reset_pqr_assoc_closid(void *v)
+{
+	struct intel_pqr_state *state = this_cpu_ptr(&pqr_state);
+
+	state->closid = 0;
+	wrmsr(MSR_IA32_PQR_ASSOC, state->rmid, 0);
+}
+
+static int intel_rdt_online_cpu(unsigned int cpu)
+{
+	struct rdt_resource *r;
+
+	mutex_lock(&rdtgroup_mutex);
+	for_each_rdt_resource(r)
+		update_domain(cpu, r, 1);
+	smp_call_function_single(cpu, rdt_reset_pqr_assoc_closid, NULL, 1);
+	mutex_unlock(&rdtgroup_mutex);
+
+	return 0;
+}
+
+static int intel_rdt_offline_cpu(unsigned int cpu)
+{
+	struct rdt_resource *r;
+
+	mutex_lock(&rdtgroup_mutex);
+	for_each_rdt_resource(r)
+		update_domain(cpu, r, 0);
+	mutex_unlock(&rdtgroup_mutex);
+
+	return 0;
+}
+
 static int __init intel_rdt_late_init(void)
 {
 	struct cpuinfo_x86 *c = &boot_cpu_data;
 	struct rdt_resource *r;
+	int ret;
 
 	if (!get_rdt_resources(c))
 		return -ENODEV;
 
+	ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
+				"AP_INTEL_RDT_ONLINE",
+				intel_rdt_online_cpu, intel_rdt_offline_cpu);
+	if (ret < 0)
+		return ret;
+
 	for_each_rdt_resource(r)
 		rdt_max_closid = max(rdt_max_closid, r->max_closid);
 
-- 
2.5.0

[toc] | [prev] | [next] | [standalone]

#1498021 — Re: [PATCH v3 10/18] x86/intel_rdt: Build structures for each resource based on cache topology

On 7 October 2016 at 21:45, Fenghua Yu <fenghua.yu@intel.com> wrote:
> From: Tony Luck <tony.luck@intel.com>
>
> diff --git a/arch/x86/include/asm/intel_rdt.h b/arch/x86/include/asm/intel_rdt.h
> index 251ac2a..bad8dc7 100644
> --- a/arch/x86/include/asm/intel_rdt.h
> +++ b/arch/x86/include/asm/intel_rdt.h
> @@ -40,6 +40,39 @@ struct rdt_resource {
>                 if (r->enabled)
>
>  #define IA32_L3_CBM_BASE       0xc90
> +
> +/**
> + * struct rdt_domain - group of cpus sharing an RDT resource
> + * @list:      all instances of this resource
> + * @id:                unique id for this instance
> + * @cpu_mask:  which cpus share this resource
> + * @cbm:       array of cache bit masks (indexed by CLOSID)
> + */
> +struct rdt_domain {
> +       struct list_head        list;
> +       int                     id;
> +       struct cpumask          cpu_mask;
> +       u32                     *cbm;
> +};
> +
> +/**
> + * struct msr_param - set a range of MSRs from a domain
> + * @res:       The resource to use
> + * @low:       Beginning index from base MSR
> + * @high:      End index
> + */
> +struct msr_param {
> +       struct rdt_resource     *res;
> +       int                     low;
> +       int                     high;
> +};
> +
> +extern struct static_key_false rdt_enable_key;
> +extern struct mutex rdtgroup_mutex;
> +
> +int __init rdtgroup_init(void);
> +
> +extern struct rdtgroup *rdtgroup_default;

struct rdtgroup has not been defined yet.  The statement above should
be part of the next patch.

--
Nilay

[toc] | [prev] | [next] | [standalone]

#1497610 — [PATCH v3 16/18] x86/intel_rdt: Add schemata file

From: Tony Luck <tony.luck@intel.com>

Last of the per resource group files. Also mode 0644. This one shows
the resources available to the group. Syntax depends on whether the
"cdp" mount option was given. With code/data prioritization disabled
it is simply a list of masks for each cache domain. Initial value
allows access to all of the L3 cache on all domains. E.g. on a 2 socket
Broadwell:
        L3:0=fffff;1=fffff
With CDP enabled, separate masks for data and instructions are provided:
        L3:0=fffff,fffff;1=fffff,fffff

Signed-off-by: Tony Luck <tony.luck@intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
---
 arch/x86/include/asm/intel_rdt.h         |   6 +
 arch/x86/kernel/cpu/Makefile             |   2 +-
 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c |   7 +
 arch/x86/kernel/cpu/intel_rdt_schemata.c | 260 +++++++++++++++++++++++++++++++
 4 files changed, 274 insertions(+), 1 deletion(-)
 create mode 100644 arch/x86/kernel/cpu/intel_rdt_schemata.c

diff --git a/arch/x86/include/asm/intel_rdt.h b/arch/x86/include/asm/intel_rdt.h
index 7c74272..c052652 100644
--- a/arch/x86/include/asm/intel_rdt.h
+++ b/arch/x86/include/asm/intel_rdt.h
@@ -67,6 +67,7 @@ struct rftype {
  * @cdp_capable:		Code/Data Prioritization available
  * @cdp_enabled:		Code/Data Prioritization enabled
  * @tmp_cbms:			Scratch space when updating schemata
+ * @num_cbms:			Number of CBMs in tmp_cbms
  * @cache_level:		Which cache level defines scope of this domain
  */
 struct rdt_resource {
@@ -83,6 +84,7 @@ struct rdt_resource {
 	bool			cdp_capable;
 	bool			cdp_enabled;
 	u32			*tmp_cbms;
+	int			num_cbms;
 	int			cache_level;
 };
 
@@ -155,4 +157,8 @@ DECLARE_PER_CPU_READ_MOSTLY(int, cpu_closid);
 void rdt_cbm_update(void *arg);
 struct rdtgroup *rdtgroup_kn_lock_live(struct kernfs_node *kn);
 void rdtgroup_kn_unlock(struct kernfs_node *kn);
+ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,
+				char *buf, size_t nbytes, loff_t off);
+int rdtgroup_schemata_show(struct kernfs_open_file *of,
+			   struct seq_file *s, void *v);
 #endif /* _ASM_X86_INTEL_RDT_H */
diff --git a/arch/x86/kernel/cpu/Makefile b/arch/x86/kernel/cpu/Makefile
index 963c54a..4d7e0e5 100644
--- a/arch/x86/kernel/cpu/Makefile
+++ b/arch/x86/kernel/cpu/Makefile
@@ -34,7 +34,7 @@ obj-$(CONFIG_CPU_SUP_CENTAUR)		+= centaur.o
 obj-$(CONFIG_CPU_SUP_TRANSMETA_32)	+= transmeta.o
 obj-$(CONFIG_CPU_SUP_UMC_32)		+= umc.o
 
-obj-$(CONFIG_INTEL_RDT)	+= intel_rdt.o intel_rdt_rdtgroup.o
+obj-$(CONFIG_INTEL_RDT)	+= intel_rdt.o intel_rdt_rdtgroup.o intel_rdt_schemata.o
 
 obj-$(CONFIG_X86_MCE)			+= mcheck/
 obj-$(CONFIG_MTRR)			+= mtrr/
diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
index e9e834b..4740b7e 100644
--- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
+++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
@@ -425,6 +425,13 @@ static struct rftype rdtgroup_base_files[] = {
 		.seq_show	= rdtgroup_tasks_show,
 	},
 	{
+		.name		= "schemata",
+		.mode		= 0644,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.write		= rdtgroup_schemata_write,
+		.seq_show	= rdtgroup_schemata_show,
+	},
+	{
 		/* NULL terminated */
 	}
 };
diff --git a/arch/x86/kernel/cpu/intel_rdt_schemata.c b/arch/x86/kernel/cpu/intel_rdt_schemata.c
new file mode 100644
index 0000000..d666854
--- /dev/null
+++ b/arch/x86/kernel/cpu/intel_rdt_schemata.c
@@ -0,0 +1,260 @@
+/*
+ * Resource Director Technology(RDT)
+ * - Cache Allocation code.
+ *
+ * Copyright (C) 2016 Intel Corporation
+ *
+ * Authors:
+ *    Fenghua Yu <fenghua.yu@intel.com>
+ *    Tony Luck <tony.luck@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * More information about RDT be found in the Intel (R) x86 Architecture
+ * Software Developer Manual June 2016, volume 3, section 17.17.
+ */
+
+#define pr_fmt(fmt)	KBUILD_MODNAME ": " fmt
+
+#include <linux/kernfs.h>
+#include <linux/seq_file.h>
+#include <linux/slab.h>
+#include <asm/intel_rdt.h>
+
+/*
+ * Check whether a cache bit mask is valid. The SDM says:
+ *	Please note that all (and only) contiguous '1' combinations
+ *	are allowed (e.g. FFFFH, 0FF0H, 003CH, etc.).
+ * Additionally Haswell requires at least two bits set.
+ */
+static bool cbm_validate(unsigned long var, struct rdt_resource *r)
+{
+	unsigned long first_bit, zero_bit;
+
+	if (var == 0 || var > r->max_cbm)
+		return false;
+
+	first_bit = find_first_bit(&var, r->cbm_len);
+	zero_bit = find_next_zero_bit(&var, r->cbm_len, first_bit);
+
+	if (find_next_bit(&var, r->cbm_len, zero_bit) < r->cbm_len)
+		return false;
+
+	if ((zero_bit - first_bit) < r->min_cbm_bits)
+		return false;
+	return true;
+}
+
+/*
+ * Read one cache bit mask (hex). Check that it is valid for the current
+ * resource type.
+ */
+static int parse_cbm_token(char *tok, struct rdt_resource *r)
+{
+	unsigned long data;
+	int ret;
+
+	ret = kstrtoul(tok, 16, &data);
+	if (ret)
+		return ret;
+	if (!cbm_validate(data, r))
+		return -EINVAL;
+	r->tmp_cbms[r->num_cbms++] = data;
+	return 0;
+}
+
+/*
+ * If code/data prioritization is enabled for this resource we need
+ * two bit masks separated by a ",". Otherwise a single bit mask.
+ */
+static int parse_cbm(char *buf, struct rdt_resource *r)
+{
+	char *cbm1 = buf;
+	int ret;
+
+	if (r->cdp_enabled)
+		cbm1 = strsep(&buf, ",");
+	if (!cbm1 || !buf)
+		return 1;
+	ret = parse_cbm_token(cbm1, r);
+	if (ret)
+		return ret;
+	if (r->cdp_enabled)
+		return parse_cbm_token(buf, r);
+	return 0;
+}
+
+/*
+ * For each domain in this resource we expect to find a series of:
+ *	id=mask[,mask]
+ * separated by ";". The "id" is in decimal, and must appear in the
+ * right order.
+ */
+static int parse_line(char *line, struct rdt_resource *r)
+{
+	struct list_head *l;
+	struct rdt_domain *d;
+	char *dom = NULL, *id;
+	unsigned long dom_id;
+
+	list_for_each(l, &r->domains) {
+		d = list_entry(l, struct rdt_domain, list);
+		dom = strsep(&line, ";");
+		if (!dom)
+			return -EINVAL;
+		id = strsep(&dom, "=");
+		if (kstrtoul(id, 10, &dom_id) || dom_id != d->id)
+			return -EINVAL;
+		if (parse_cbm(dom, r))
+			return -EINVAL;
+	}
+
+	/* Any garbage at the end of the line? */
+	if (line && line[0])
+		return -EINVAL;
+	return 0;
+}
+
+static void update_domains(struct rdt_resource *r, int closid)
+{
+	int idx = 0;
+	struct list_head *l;
+	struct rdt_domain *d;
+	struct msr_param msr_param;
+	struct cpumask cpu_mask;
+
+	cpumask_clear(&cpu_mask);
+	msr_param.low = closid << r->cdp_enabled;
+	msr_param.high = msr_param.low + 1 + r->cdp_enabled;
+	msr_param.res = r;
+
+	list_for_each(l, &r->domains) {
+		d = list_entry(l, struct rdt_domain, list);
+		cpumask_set_cpu(cpumask_any(&d->cpu_mask), &cpu_mask);
+		d->cbm[msr_param.low] = r->tmp_cbms[idx++];
+		if (r->cdp_enabled)
+			d->cbm[msr_param.low + 1] = r->tmp_cbms[idx++];
+	}
+	smp_call_function_many(&cpu_mask, rdt_cbm_update, &msr_param, 1);
+}
+
+ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,
+				char *buf, size_t nbytes, loff_t off)
+{
+	char *tok, *resname;
+	struct rdtgroup *rdtgrp;
+	struct rdt_resource *r;
+	int closid, ret = 0;
+	u32 *l3_cbms = NULL;
+
+	/* Legal input requires a trailing newline */
+	if (nbytes == 0 || buf[nbytes - 1] != '\n')
+		return -EINVAL;
+	buf[nbytes - 1] = '\0';
+
+	rdtgrp = rdtgroup_kn_lock_live(of->kn);
+	if (!rdtgrp) {
+		rdtgroup_kn_unlock(of->kn);
+		return -ENOENT;
+	}
+
+	closid = rdtgrp->closid;
+
+	/* get scratch space to save all the masks while we validate input */
+	for_each_rdt_resource(r) {
+		r->tmp_cbms = kcalloc(r->num_domains << r->cdp_enabled,
+				      sizeof(*l3_cbms), GFP_KERNEL);
+		if (!r->tmp_cbms) {
+			ret = -ENOMEM;
+			goto fail;
+		}
+		r->num_cbms = 0;
+	}
+
+	while ((tok = strsep(&buf, "\n")) != NULL) {
+		resname = strsep(&tok, ":");
+		if (!tok) {
+			ret = -EINVAL;
+			goto fail;
+		}
+		for_each_rdt_resource(r) {
+			if (!strcmp(resname, r->name) &&
+			    closid < r->num_closid) {
+				ret = parse_line(tok, r);
+				if (ret)
+					goto fail;
+				break;
+			}
+		}
+		if (!r->name) {
+			ret = -EINVAL;
+			goto fail;
+		}
+	}
+
+	/* Did the parser find all the masks we need? */
+	for_each_rdt_resource(r) {
+		if (r->num_cbms != r->num_domains << r->cdp_enabled) {
+			ret = -EINVAL;
+			goto fail;
+		}
+	}
+
+	for_each_rdt_resource(r)
+		update_domains(r, closid);
+
+fail:
+	rdtgroup_kn_unlock(of->kn);
+	for_each_rdt_resource(r) {
+		kfree(r->tmp_cbms);
+		r->tmp_cbms = NULL;
+	}
+	return ret ?: nbytes;
+}
+
+static void show_doms(struct seq_file *s, struct rdt_resource *r, int closid)
+{
+	struct list_head *l;
+	struct rdt_domain *dom;
+	int idx = closid << r->cdp_enabled;
+	bool sep = false;
+
+	seq_printf(s, "%s:", r->name);
+	list_for_each(l, &r->domains) {
+		dom = list_entry(l, struct rdt_domain, list);
+		if (sep)
+			seq_puts(s, ";");
+		seq_printf(s, "%d=%x", dom->id, dom->cbm[idx]);
+		if (r->cdp_enabled)
+			seq_printf(s, ",%x", dom->cbm[idx + 1]);
+		sep = true;
+	}
+	seq_puts(s, "\n");
+}
+
+int rdtgroup_schemata_show(struct kernfs_open_file *of,
+			   struct seq_file *s, void *v)
+{
+	struct rdtgroup *rdtgrp;
+	struct rdt_resource *r;
+	int closid, ret = 0;
+
+	rdtgrp = rdtgroup_kn_lock_live(of->kn);
+	if (rdtgrp) {
+		closid = rdtgrp->closid;
+		for_each_rdt_resource(r)
+			if (closid < r->num_closid)
+				show_doms(s, r, closid);
+	} else {
+		ret = -ENOENT;
+	}
+	rdtgroup_kn_unlock(of->kn);
+	return ret;
+}
-- 
2.5.0

[toc] | [prev] | [next] | [standalone]

#1497611 — [PATCH v3 07/18] x86/intel_rdt: Add Haswell feature discovery

From: Fenghua Yu <fenghua.yu@intel.com>

Some Haswell generation CPUs support RDT, but they don't enumerate this
using CPUID.  Use rdmsr_safe() and wrmsr_safe() to probe the MSRs on
cpu model 63 (INTEL_FAM6_HASWELL_X)

Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Signed-off-by: Tony Luck <tony.luck@intel.com>
---
 arch/x86/events/intel/cqm.c             |  2 +-
 arch/x86/include/asm/intel_rdt_common.h |  6 ++++++
 arch/x86/kernel/cpu/intel_rdt.c         | 38 +++++++++++++++++++++++++++++++++
 3 files changed, 45 insertions(+), 1 deletion(-)
 create mode 100644 arch/x86/include/asm/intel_rdt_common.h

diff --git a/arch/x86/events/intel/cqm.c b/arch/x86/events/intel/cqm.c
index 8f82b02..df86874 100644
--- a/arch/x86/events/intel/cqm.c
+++ b/arch/x86/events/intel/cqm.c
@@ -7,9 +7,9 @@
 #include <linux/perf_event.h>
 #include <linux/slab.h>
 #include <asm/cpu_device_id.h>
+#include <asm/intel_rdt_common.h>
 #include "../perf_event.h"
 
-#define MSR_IA32_PQR_ASSOC	0x0c8f
 #define MSR_IA32_QM_CTR		0x0c8e
 #define MSR_IA32_QM_EVTSEL	0x0c8d
 
diff --git a/arch/x86/include/asm/intel_rdt_common.h b/arch/x86/include/asm/intel_rdt_common.h
new file mode 100644
index 0000000..e6e15cf
--- /dev/null
+++ b/arch/x86/include/asm/intel_rdt_common.h
@@ -0,0 +1,6 @@
+#ifndef _ASM_X86_INTEL_RDT_COMMON_H
+#define _ASM_X86_INTEL_RDT_COMMON_H
+
+#define MSR_IA32_PQR_ASSOC	0x0c8f
+
+#endif /* _ASM_X86_INTEL_RDT_COMMON_H */
diff --git a/arch/x86/kernel/cpu/intel_rdt.c b/arch/x86/kernel/cpu/intel_rdt.c
index ebe8dae..bc7f10b 100644
--- a/arch/x86/kernel/cpu/intel_rdt.c
+++ b/arch/x86/kernel/cpu/intel_rdt.c
@@ -26,11 +26,49 @@
 
 #include <linux/slab.h>
 #include <linux/err.h>
+#include <asm/intel_rdt_common.h>
+#include <asm/intel-family.h>
+
+/*
+ * cache_alloc_hsw_probe() - Have to probe for Intel haswell server CPUs
+ * as it does not have CPUID enumeration support for Cache allocation.
+ *
+ * Probes by writing to the high 32 bits(CLOSid) of the IA32_PQR_MSR and
+ * testing if the bits stick. Max CLOSids is always 4 and max cbm length
+ * is always 20 on hsw server parts. The minimum cache bitmask length
+ * allowed for HSW server is always 2 bits. Hardcode all of them.
+ */
+static inline bool cache_alloc_hsw_probe(void)
+{
+	u32 l, h_old, h_new, h_tmp;
+
+	if (rdmsr_safe(MSR_IA32_PQR_ASSOC, &l, &h_old))
+		return false;
+
+	/*
+	 * Default value is always 0 if feature is present.
+	 */
+	h_tmp = h_old ^ 0x1U;
+	if (wrmsr_safe(MSR_IA32_PQR_ASSOC, l, h_tmp))
+		return false;
+	rdmsr(MSR_IA32_PQR_ASSOC, l, h_new);
+
+	if (h_tmp != h_new)
+		return false;
+
+	wrmsr(MSR_IA32_PQR_ASSOC, l, h_old);
+
+	return true;
+}
 
 static inline bool get_rdt_resources(struct cpuinfo_x86 *c)
 {
 	bool ret = false;
 
+	if (c->x86_vendor == X86_VENDOR_INTEL && c->x86 == 6 &&
+	    c->x86_model == INTEL_FAM6_HASWELL_X)
+		return cache_alloc_hsw_probe();
+
 	if (!cpu_has(c, X86_FEATURE_RDT_A))
 		return false;
 	if (cpu_has(c, X86_FEATURE_CAT_L3))
-- 
2.5.0

[toc] | [prev] | [next] | [standalone]

#1497901 — Re: [PATCH v3 07/18] x86/intel_rdt: Add Haswell feature discovery

On Fri, Oct 07, 2016 at 07:45:52PM -0700, Fenghua Yu wrote:
> From: Fenghua Yu <fenghua.yu@intel.com>
> 
> Some Haswell generation CPUs support RDT, but they don't enumerate this
> using CPUID.  Use rdmsr_safe() and wrmsr_safe() to probe the MSRs on
> cpu model 63 (INTEL_FAM6_HASWELL_X)
> 
> Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
> Signed-off-by: Tony Luck <tony.luck@intel.com>
> ---
>  arch/x86/events/intel/cqm.c             |  2 +-
>  arch/x86/include/asm/intel_rdt_common.h |  6 ++++++
>  arch/x86/kernel/cpu/intel_rdt.c         | 38 +++++++++++++++++++++++++++++++++
>  3 files changed, 45 insertions(+), 1 deletion(-)
>  create mode 100644 arch/x86/include/asm/intel_rdt_common.h

...

> +static inline bool cache_alloc_hsw_probe(void)
> +{
> +	u32 l, h_old, h_new, h_tmp;
> +
> +	if (rdmsr_safe(MSR_IA32_PQR_ASSOC, &l, &h_old))
> +		return false;
> +
> +	/*
> +	 * Default value is always 0 if feature is present.
> +	 */
> +	h_tmp = h_old ^ 0x1U;
> +	if (wrmsr_safe(MSR_IA32_PQR_ASSOC, l, h_tmp))

I don't understand - you do the family/model check below and yet still
use the _safe() variants. Isn't the presence of that MSR guaranteed on
those machines?

> +		return false;
> +	rdmsr(MSR_IA32_PQR_ASSOC, l, h_new);
> +
> +	if (h_tmp != h_new)
> +		return false;
> +
> +	wrmsr(MSR_IA32_PQR_ASSOC, l, h_old);
> +
> +	return true;
> +}
>  
>  static inline bool get_rdt_resources(struct cpuinfo_x86 *c)
>  {
>  	bool ret = false;
>  
> +	if (c->x86_vendor == X86_VENDOR_INTEL && c->x86 == 6 &&
> +	    c->x86_model == INTEL_FAM6_HASWELL_X)
> +		return cache_alloc_hsw_probe();
> +
>  	if (!cpu_has(c, X86_FEATURE_RDT_A))
>  		return false;
>  	if (cpu_has(c, X86_FEATURE_CAT_L3))
> -- 
> 2.5.0
> 

-- 
Regards/Gruss,
    Boris.

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
-- 

[toc] | [prev] | [next] | [standalone]

#1497905 — Re: [PATCH v3 07/18] x86/intel_rdt: Add Haswell feature discovery

On Sun, Oct 09, 2016 at 01:41:16PM +0200, Borislav Petkov wrote:
> On Fri, Oct 07, 2016 at 07:45:52PM -0700, Fenghua Yu wrote:
> > From: Fenghua Yu <fenghua.yu@intel.com>
> > 
> > Some Haswell generation CPUs support RDT, but they don't enumerate this
> > using CPUID.  Use rdmsr_safe() and wrmsr_safe() to probe the MSRs on
> > cpu model 63 (INTEL_FAM6_HASWELL_X)
> > 
> > Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
> > Signed-off-by: Tony Luck <tony.luck@intel.com>
> > ---
> >  arch/x86/events/intel/cqm.c             |  2 +-
> >  arch/x86/include/asm/intel_rdt_common.h |  6 ++++++
> >  arch/x86/kernel/cpu/intel_rdt.c         | 38 +++++++++++++++++++++++++++++++++
> >  3 files changed, 45 insertions(+), 1 deletion(-)
> >  create mode 100644 arch/x86/include/asm/intel_rdt_common.h
> 
> ...
> 
> > +static inline bool cache_alloc_hsw_probe(void)
> > +{
> > +	u32 l, h_old, h_new, h_tmp;
> > +
> > +	if (rdmsr_safe(MSR_IA32_PQR_ASSOC, &l, &h_old))
> > +		return false;
> > +
> > +	/*
> > +	 * Default value is always 0 if feature is present.
> > +	 */
> > +	h_tmp = h_old ^ 0x1U;
> > +	if (wrmsr_safe(MSR_IA32_PQR_ASSOC, l, h_tmp))
> 
> I don't understand - you do the family/model check below and yet still
> use the _safe() variants. Isn't the presence of that MSR guaranteed on
> those machines?

The MSR is not guaranteed on every stepping of the family and model machine
because some parts may have the MSR fused off. And some bits in the MSR
may not be implemented on some parts. And in KVM or guest, the MSR may not
implemented. Those are reasons why we use wrmsr_safe/rdmsr_safe in Haswell
probe.

Thanks.

-Fenghua

[toc] | [prev] | [next] | [standalone]

#1497976 — Re: [PATCH v3 07/18] x86/intel_rdt: Add Haswell feature discovery

On Sun, Oct 09, 2016 at 10:09:37AM -0700, Fenghua Yu wrote:
> The MSR is not guaranteed on every stepping of the family and model machine
> because some parts may have the MSR fused off. And some bits in the MSR
> may not be implemented on some parts. And in KVM or guest, the MSR may not
> implemented. Those are reasons why we use wrmsr_safe/rdmsr_safe in Haswell
> probe.

Please add that info in a comment somewhere there as we'll all forget
about it otherwise.

Thanks.

-- 
Regards/Gruss,
    Boris.

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
-- 

[toc] | [prev] | [next] | [standalone]

#1498481 — Re: [PATCH v3 07/18] x86/intel_rdt: Add Haswell feature discovery

On Sun, Oct 09, 2016 at 06:28:23PM +0200, Borislav Petkov wrote:
> On Sun, Oct 09, 2016 at 10:09:37AM -0700, Fenghua Yu wrote:
> > The MSR is not guaranteed on every stepping of the family and model machine
> > because some parts may have the MSR fused off. And some bits in the MSR
> > may not be implemented on some parts. And in KVM or guest, the MSR may not
> > implemented. Those are reasons why we use wrmsr_safe/rdmsr_safe in Haswell
> > probe.
> 
> Please add that info in a comment somewhere there as we'll all forget
> about it otherwise.

How about this (this diff on top of current series, but obviously we'll
fold it into part 07.


commit cdb05159fb91ed1f85c950c0f2c6de25f143961d
Author: Tony Luck <tony.luck@intel.com>
Date:   Mon Oct 10 11:48:42 2016 -0700

    Update the HSW probe code - better comments, and use IA32_L3_CBM_BASE
    as the probe MSR instead of PQR_ASSOC at suggestion of h/w architect).

diff --git a/arch/x86/kernel/cpu/intel_rdt.c b/arch/x86/kernel/cpu/intel_rdt.c
index 4903e21d660d..e3c397306f1a 100644
--- a/arch/x86/kernel/cpu/intel_rdt.c
+++ b/arch/x86/kernel/cpu/intel_rdt.c
@@ -56,39 +56,39 @@ struct rdt_resource rdt_resources_all[] = {
 
 /*
  * cache_alloc_hsw_probe() - Have to probe for Intel haswell server CPUs
- * as it does not have CPUID enumeration support for Cache allocation.
+ * as they do not have CPUID enumeration support for Cache allocation.
+ * The check for Vendor/Family/Model is not enough to guarantee that
+ * the MSRs won't #GP fault because only the following SKUs support
+ * CAT:
+ *	Intel(R) Xeon(R)  CPU E5-2658 v3  @  2.20GHz
+ *	Intel(R) Xeon(R)  CPU E5-2648L v3  @  1.80GHz
+ *	Intel(R) Xeon(R)  CPU E5-2628L v3  @  2.00GHz
+ *	Intel(R) Xeon(R)  CPU E5-2618L v3  @  2.30GHz
+ *	Intel(R) Xeon(R)  CPU E5-2608L v3  @  2.00GHz
  *
- * Probes by writing to the high 32 bits(CLOSid) of the IA32_PQR_MSR and
- * testing if the bits stick. Max CLOSids is always 4 and max cbm length
+ * Probe by trying to write the first of the L3 cach mask registers
+ * and checking that the bits stick. Max CLOSids is always 4 and max cbm length
  * is always 20 on hsw server parts. The minimum cache bitmask length
  * allowed for HSW server is always 2 bits. Hardcode all of them.
  */
 static inline bool cache_alloc_hsw_probe(void)
 {
-	u32 l, h_old, h_new, h_tmp;
+	u32 l, h;
 	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3];
 
-	if (rdmsr_safe(MSR_IA32_PQR_ASSOC, &l, &h_old))
-		return false;
-
-	/*
-	 * Default value is always 0 if feature is present.
-	 */
-	h_tmp = h_old ^ 0x1U;
-	if (wrmsr_safe(MSR_IA32_PQR_ASSOC, l, h_tmp))
-		return false;
-	rdmsr(MSR_IA32_PQR_ASSOC, l, h_new);
-
-	if (h_tmp != h_new)
-		return false;
-
-	wrmsr(MSR_IA32_PQR_ASSOC, l, h_old);
-
 	r->max_closid = 4;
 	r->num_closid = r->max_closid;
 	r->cbm_len = 20;
 	r->max_cbm = BIT_MASK(20) - 1;
 	r->min_cbm_bits = 2;
+
+	if (wrmsr_safe(IA32_L3_CBM_BASE, r->max_cbm, 0))
+		return false;
+	rdmsr(IA32_L3_CBM_BASE, l, h);
+
+	if (l != r->max_cbm)
+		return false;
+
 	r->enabled = true;
 
 	return true;

[toc] | [prev] | [next] | [standalone]

#1498761 — Re: [PATCH v3 07/18] x86/intel_rdt: Add Haswell feature discovery

On Mon, Oct 10, 2016 at 11:55:45AM -0700, Luck, Tony wrote:
> How about this (this diff on top of current series, but obviously we'll
> fold it into part 07.
> 
> 
> commit cdb05159fb91ed1f85c950c0f2c6de25f143961d
> Author: Tony Luck <tony.luck@intel.com>
> Date:   Mon Oct 10 11:48:42 2016 -0700
> 
>     Update the HSW probe code - better comments, and use IA32_L3_CBM_BASE
>     as the probe MSR instead of PQR_ASSOC at suggestion of h/w architect).
> 
> diff --git a/arch/x86/kernel/cpu/intel_rdt.c b/arch/x86/kernel/cpu/intel_rdt.c
> index 4903e21d660d..e3c397306f1a 100644
> --- a/arch/x86/kernel/cpu/intel_rdt.c
> +++ b/arch/x86/kernel/cpu/intel_rdt.c
> @@ -56,39 +56,39 @@ struct rdt_resource rdt_resources_all[] = {
>  
>  /*
>   * cache_alloc_hsw_probe() - Have to probe for Intel haswell server CPUs
> - * as it does not have CPUID enumeration support for Cache allocation.
> + * as they do not have CPUID enumeration support for Cache allocation.
> + * The check for Vendor/Family/Model is not enough to guarantee that
> + * the MSRs won't #GP fault because only the following SKUs support
> + * CAT:
> + *	Intel(R) Xeon(R)  CPU E5-2658 v3  @  2.20GHz
> + *	Intel(R) Xeon(R)  CPU E5-2648L v3  @  1.80GHz
> + *	Intel(R) Xeon(R)  CPU E5-2628L v3  @  2.00GHz
> + *	Intel(R) Xeon(R)  CPU E5-2618L v3  @  2.30GHz
> + *	Intel(R) Xeon(R)  CPU E5-2608L v3  @  2.00GHz
>   *
> - * Probes by writing to the high 32 bits(CLOSid) of the IA32_PQR_MSR and
> - * testing if the bits stick. Max CLOSids is always 4 and max cbm length
> + * Probe by trying to write the first of the L3 cach mask registers
> + * and checking that the bits stick. Max CLOSids is always 4 and max cbm length

I wonder what's worse - comparing SKU strings - we know that from the MCE
recovery experience - or poking at maybe nonexistent MSRs? :-)

I guess the latter is cleaner so let's try it.

Thanks for the writeup in the comments - this is exactly what I was
thinking about!

-- 
Regards/Gruss,
    Boris.

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
-- 

[toc] | [prev] | [next] | [standalone]

#1498949 — RE: [PATCH v3 07/18] x86/intel_rdt: Add Haswell feature discovery

> I wonder what's worse - comparing SKU strings - we know that from the MCE
> recovery experience - or poking at maybe nonexistent MSRs? :-)
>
> I guess the latter is cleaner so let's try it.

Vikas got beat up for comparing SKU strings, so the probe method
was offered as an alternative.  It's definitely more robust. E.g. my
list turns out to be incomplete, somehow I forgot to include:

    Intel(R) Xeon(R) processor E5-2658A v3

-Tony
 

[toc] | [prev] | [next] | [standalone]

#1497613 — [PATCH v3 13/18] x86/intel_rdt: Add mkdir to resctrl file system

From: Fenghua Yu <fenghua.yu@intel.com>

Resource control groups are represented as directories in the resctrl
file system. The root directory describes the default resources available
to tasks that have not been assigned specific resources. Other directories
can be created at the root level to make new resource groups. It is not
permitted to make directories within other directories.

Hardware uses a CLOSID (Class of service ID) to determine which resource
limits are currently in effect. The exact number available is enumerated
by CPUID leaf 0x10, but on current implementations it is a small number.
We implement a simple bitmask allocator for CLOSIDs.

Each resource control group uses one CLOSID, which limits the total number
of directories that can be created.

Resource groups can be removed using rmdir.

Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Signed-off-by: Tony Luck <tony.luck@intel.com>
---
 arch/x86/include/asm/intel_rdt.h         |   9 ++
 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 222 +++++++++++++++++++++++++++++++
 2 files changed, 231 insertions(+)

diff --git a/arch/x86/include/asm/intel_rdt.h b/arch/x86/include/asm/intel_rdt.h
index 8e3fafc..99d8d14 100644
--- a/arch/x86/include/asm/intel_rdt.h
+++ b/arch/x86/include/asm/intel_rdt.h
@@ -6,13 +6,20 @@
  * @kn:				kernfs node
  * @rdtgroup_list:		linked list for all rdtgroups
  * @closid:			closid for this rdtgroup
+ * @flags:			status bits
+ * @waitcount:			how many cpus expect to find this
  */
 struct rdtgroup {
 	struct kernfs_node	*kn;
 	struct list_head	rdtgroup_list;
 	int			closid;
+	int			flags;
+	atomic_t		waitcount;
 };
 
+/* rdtgroup.flags */
+#define	RDT_DELETED		1
+
 /* List of all resource groups */
 extern struct list_head rdt_all_groups;
 
@@ -141,4 +148,6 @@ union cpuid_0x10_1_edx {
 };
 
 void rdt_cbm_update(void *arg);
+struct rdtgroup *rdtgroup_kn_lock_live(struct kernfs_node *kn);
+void rdtgroup_kn_unlock(struct kernfs_node *kn);
 #endif /* _ASM_X86_INTEL_RDT_H */
diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
index 28df92e..efcbfe7 100644
--- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
+++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
@@ -36,6 +36,52 @@ struct kernfs_root *rdt_root;
 struct rdtgroup rdtgroup_default;
 LIST_HEAD(rdt_all_groups);
 
+/*
+ * Trivial allocator for CLOSIDs. Since h/w only supports
+ * a small number, we can keep a bitmap of free CLOSIDs in
+ * a single integer.
+ */
+static int closid_free_map;
+
+static void closid_init(void)
+{
+	closid_free_map = BIT_MASK(rdt_max_closid) - 1;
+
+	/* CLOSID 0 is always reserved for the default group */
+	closid_free_map &= ~1;
+}
+
+int closid_alloc(void)
+{
+	int closid = ffs(closid_free_map);
+
+	if (closid == 0)
+		return -ENOSPC;
+	closid--;
+	closid_free_map &= ~(1 << closid);
+
+	return closid;
+}
+
+static void closid_free(int closid)
+{
+	closid_free_map |= 1 << closid;
+}
+
+static struct rdtgroup *rdtgroup_alloc(void)
+{
+	struct rdtgroup *rdtgrp;
+
+	rdtgrp = kzalloc(sizeof(*rdtgrp), GFP_KERNEL);
+
+	return rdtgrp;
+}
+
+static void rdtgroup_free(struct rdtgroup *rdtgroup)
+{
+	kfree(rdtgroup);
+}
+
 /* set uid and gid of rdtgroup dirs and files to that of the creator */
 static int rdtgroup_kn_set_ugid(struct kernfs_node *kn)
 {
@@ -240,6 +286,54 @@ static int parse_rdtgroupfs_options(char *data, struct rdt_resource *r)
 	return 0;
 }
 
+/*
+ * We don't allow rdtgroup directories to be created anywhere
+ * except the root directory. Thus when looking for the rdtgroup
+ * structure for a kernfs node we are either looking at a directory,
+ * in which case the rdtgroup structure is pointed at by the "priv"
+ * field, otherwise we have a file, and need only look to the parent
+ * to find the rdtgroup.
+ */
+static struct rdtgroup *kernfs_to_rdtgroup(struct kernfs_node *kn)
+{
+	if (kernfs_type(kn) == KERNFS_DIR)
+		return kn->priv;
+	else
+		return kn->parent->priv;
+}
+
+struct rdtgroup *rdtgroup_kn_lock_live(struct kernfs_node *kn)
+{
+	struct rdtgroup *rdtgrp = kernfs_to_rdtgroup(kn);
+
+	atomic_inc(&rdtgrp->waitcount);
+	kernfs_break_active_protection(kn);
+
+	mutex_lock(&rdtgroup_mutex);
+
+	/* Was this group deleted while we waited? */
+	if (rdtgrp->flags & RDT_DELETED)
+		return NULL;
+
+	return rdtgrp;
+}
+
+void rdtgroup_kn_unlock(struct kernfs_node *kn)
+{
+	struct rdtgroup *rdtgrp = kernfs_to_rdtgroup(kn);
+
+	mutex_unlock(&rdtgroup_mutex);
+
+	if (atomic_dec_and_test(&rdtgrp->waitcount) &&
+	    (rdtgrp->flags & RDT_DELETED)) {
+		kernfs_unbreak_active_protection(kn);
+		kernfs_put(kn);
+		kfree(rdtgrp);
+	} else {
+		kernfs_unbreak_active_protection(kn);
+	}
+}
+
 static struct dentry *rdt_mount(struct file_system_type *fs_type,
 				int flags, const char *unused_dev_name,
 				void *data)
@@ -275,6 +369,7 @@ static struct dentry *rdt_mount(struct file_system_type *fs_type,
 		rdt_max_closid = max(rdt_max_closid, r->num_closid);
 	if (rdt_max_closid > 32)
 		rdt_max_closid = 32;
+	closid_init();
 
 	dentry = kernfs_mount(fs_type, flags, rdt_root,
 			      RDTGROUP_SUPER_MAGIC, &new_sb);
@@ -318,6 +413,24 @@ static void reset_all_cbms(struct rdt_resource *r)
 	smp_call_function_many(&cpu_mask, rdt_cbm_update, &msr_param, 1);
 }
 
+/*
+ * Forcibly remove all of subdirectories under root.
+ */
+static void rmdir_all_sub(void)
+{
+	struct rdtgroup *rdtgrp;
+	struct list_head *l, *next;
+
+	list_for_each_safe(l, next, &rdt_all_groups) {
+		rdtgrp = list_entry(l, struct rdtgroup, rdtgroup_list);
+		if (rdtgrp == &rdtgroup_default)
+			continue;
+		kernfs_remove(rdtgrp->kn);
+		list_del(&rdtgrp->rdtgroup_list);
+		rdtgroup_free(rdtgrp);
+	}
+}
+
 static void rdt_kill_sb(struct super_block *sb)
 {
 	struct rdt_resource *r;
@@ -333,6 +446,7 @@ static void rdt_kill_sb(struct super_block *sb)
 		set_l3_qos_cfg(r);
 	}
 
+	rmdir_all_sub();
 	static_branch_disable(&rdt_enable_key);
 	kernfs_kill_sb(sb);
 	mutex_unlock(&rdtgroup_mutex);
@@ -344,7 +458,115 @@ static struct file_system_type rdt_fs_type = {
 	.kill_sb = rdt_kill_sb,
 };
 
+static int rdtgroup_mkdir(struct kernfs_node *parent_kn, const char *name,
+			  umode_t mode)
+{
+	struct rdtgroup *parent, *rdtgrp;
+	struct kernfs_node *kn;
+	int ret, closid;
+
+	/* Only allow mkdir in the root directory */
+	if (parent_kn != rdtgroup_default.kn)
+		return -EPERM;
+
+	/* Do not accept '\n' to avoid unparsable situation. */
+	if (strchr(name, '\n'))
+		return -EINVAL;
+
+	parent = rdtgroup_kn_lock_live(parent_kn);
+	if (!parent) {
+		ret = -ENODEV;
+		goto out_unlock;
+	}
+
+	ret = closid_alloc();
+	if (ret < 0)
+		goto out_unlock;
+	closid = ret;
+
+	/* allocate the rdtgroup. */
+	rdtgrp = rdtgroup_alloc();
+	if (!rdtgrp) {
+		ret = -ENOSPC;
+		goto out_closid_free;
+	}
+	rdtgrp->closid = closid;
+	list_add(&rdtgrp->rdtgroup_list, &rdt_all_groups);
+
+	/* kernfs creates the directory for rdtgrp */
+	kn = kernfs_create_dir(parent->kn, name, mode, rdtgrp);
+	if (IS_ERR(kn)) {
+		ret = PTR_ERR(kn);
+		goto out_cancel_ref;
+	}
+	rdtgrp->kn = kn;
+
+	/*
+	 * This extra ref will be put in kernfs_remove() and guarantees
+	 * that @rdtgrp->kn is always accessible.
+	 */
+	kernfs_get(kn);
+
+	ret = rdtgroup_kn_set_ugid(kn);
+	if (ret)
+		goto out_destroy;
+
+	kernfs_activate(kn);
+
+	ret = 0;
+	goto out_unlock;
+
+out_destroy:
+	kernfs_remove(rdtgrp->kn);
+out_cancel_ref:
+	rdtgroup_free(rdtgrp);
+out_closid_free:
+	closid_free(closid);
+out_unlock:
+	rdtgroup_kn_unlock(parent_kn);
+	return ret;
+}
+
+static int rdtgroup_rmdir(struct kernfs_node *kn)
+{
+	struct rdtgroup *rdtgrp;
+	int ret = 0;
+
+	rdtgrp = rdtgroup_kn_lock_live(kn);
+	if (!rdtgrp) {
+		rdtgroup_kn_unlock(kn);
+		return -ENOENT;
+	}
+
+	/*
+	 * rmdir is for deleting resource groups. Don't
+	 * allow deletion of "info" or any of its subdirectories
+	 */
+	if (!rdtgrp) {
+		mutex_unlock(&rdtgroup_mutex);
+		kernfs_unbreak_active_protection(kn);
+		return -EPERM;
+	}
+
+	rdtgrp->flags = RDT_DELETED;
+	closid_free(rdtgrp->closid);
+	list_del(&rdtgrp->rdtgroup_list);
+
+	/*
+	 * one extra hold on this, will drop when we kfree(rdtgrp)
+	 * in rdtgroup_kn_unlock()
+	 */
+	kernfs_get(kn);
+	kernfs_remove(rdtgrp->kn);
+
+	rdtgroup_kn_unlock(kn);
+
+	return ret;
+}
+
 static struct kernfs_syscall_ops rdtgroup_kf_syscall_ops = {
+	.mkdir	= rdtgroup_mkdir,
+	.rmdir	= rdtgroup_rmdir,
 };
 
 static int __init rdtgroup_setup_root(void)
-- 
2.5.0

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | linux.kernel

csiph-web