Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1501214 > unrolled thread

[PATCH v4 00/18] Intel Cache Allocation Technology

Back to article view | Back to linux.kernel

Contents

  [PATCH v4 00/18] Intel Cache Allocation Technology "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
    [PATCH v4 02/18] cacheinfo: Introduce cache id "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
      Re: [PATCH v4 02/18] cacheinfo: Introduce cache id Thomas Gleixner <tglx@linutronix.de> - 2016-10-17 12:40 +0200
    [PATCH v4 16/18] x86/intel_rdt: Add schemata file "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
      Re: [PATCH v4 16/18] x86/intel_rdt: Add schemata file Thomas Gleixner <tglx@linutronix.de> - 2016-10-18 00:40 +0200
    [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file system "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
      Re: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file  system Thomas Gleixner <tglx@linutronix.de> - 2016-10-17 23:20 +0200
        Re: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file system "Luck, Tony" <tony.luck@intel.com> - 2016-10-18 00:00 +0200
          Re: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file  system Thomas Gleixner <tglx@linutronix.de> - 2016-10-18 01:00 +0200
            RE: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file  system Thomas Gleixner <tglx@linutronix.de> - 2016-10-18 01:10 +0200
              RE: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file system "Luck, Tony" <tony.luck@intel.com> - 2016-10-18 01:20 +0200
                RE: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file  system Thomas Gleixner <tglx@linutronix.de> - 2016-10-18 01:30 +0200
            RE: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file system "Luck, Tony" <tony.luck@intel.com> - 2016-10-18 01:10 +0200
        Re: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file system Fenghua Yu <fenghua.yu@intel.com> - 2016-10-18 00:20 +0200
          Re: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file  system Thomas Gleixner <tglx@linutronix.de> - 2016-10-18 01:30 +0200
            Re: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file system "Luck, Tony" <tony.luck@intel.com> - 2016-10-18 01:40 +0200
              Re: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file system Fenghua Yu <fenghua.yu@intel.com> - 2016-10-18 02:00 +0200
              Re: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file  system Thomas Gleixner <tglx@linutronix.de> - 2016-10-18 12:50 +0200
    [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters from CPUID "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
      Re: [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters  from CPUID Thomas Gleixner <tglx@linutronix.de> - 2016-10-17 15:50 +0200
        Re: [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters  from CPUID Fenghua Yu <fenghua.yu@intel.com> - 2016-10-17 17:10 +0200
          Re: [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters  from CPUID "Luck, Tony" <tony.luck@intel.com> - 2016-10-17 18:40 +0200
            RE: [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters  from CPUID "Yu, Fenghua" <fenghua.yu@intel.com> - 2016-10-17 18:50 +0200
              Re: [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters  from CPUID "Luck, Tony" <tony.luck@intel.com> - 2016-10-17 22:30 +0200
            Re: [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters  from CPUID Thomas Gleixner <tglx@linutronix.de> - 2016-10-17 19:10 +0200
          Re: [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters  from CPUID Thomas Gleixner <tglx@linutronix.de> - 2016-10-17 19:10 +0200
          Re: [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters  from CPUID Thomas Gleixner <tglx@linutronix.de> - 2016-10-17 19:10 +0200
            Re: [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters  from CPUID Fenghua Yu <fenghua.yu@intel.com> - 2016-10-17 20:20 +0200
    [PATCH v4 01/18] Documentation, ABI: Add a document entry for cache id "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
      Re: [PATCH v4 01/18] Documentation, ABI: Add a document entry for  cache id Thomas Gleixner <tglx@linutronix.de> - 2016-10-17 12:40 +0200
    [PATCH v4 18/18] MAINTAINERS: Add maintainer for Intel RDT resource allocation "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
    [PATCH v4 04/18] x86/intel_rdt: Feature discovery "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
    [PATCH v4 11/18] x86/intel_rdt: Add basic resctrl filesystem support "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
      Re: [PATCH v4 11/18] x86/intel_rdt: Add basic resctrl filesystem  support Thomas Gleixner <tglx@linutronix.de> - 2016-10-17 21:40 +0200
    [PATCH v4 05/18] Documentation, x86: Documentation for Intel resource allocation user interface "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
    [PATCH v4 17/18] x86/intel_rdt: Add scheduler hook "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
    [PATCH v4 14/18] x86/intel_rdt: Add cpus file "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
      Re: [PATCH v4 14/18] x86/intel_rdt: Add cpus file Thomas Gleixner <tglx@linutronix.de> - 2016-10-17 23:40 +0200
    [PATCH v4 12/18] x86/intel_rdt: Add "info" files to resctrl file system "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
      Re: [PATCH v4 12/18] x86/intel_rdt: Add "info" files to resctrl file  system Thomas Gleixner <tglx@linutronix.de> - 2016-10-17 21:50 +0200
    [PATCH v4 15/18] x86/intel_rdt: Add tasks files "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
      Re: [PATCH v4 15/18] x86/intel_rdt: Add tasks files Thomas Gleixner <tglx@linutronix.de> - 2016-10-18 00:10 +0200
        Re: [PATCH v4 15/18] x86/intel_rdt: Add tasks files "Luck, Tony" <tony.luck@intel.com> - 2016-10-18 00:20 +0200
    [PATCH v4 10/18] x86/intel_rdt: Build structures for each resource based on cache topology "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
      Re: [PATCH v4 10/18] x86/intel_rdt: Build structures for each resource  based on cache topology Thomas Gleixner <tglx@linutronix.de> - 2016-10-17 16:50 +0200
    [PATCH v4 03/18] x86, intel_cacheinfo: Enable cache id in x86 "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
      Re: [PATCH v4 03/18] x86, intel_cacheinfo: Enable cache id in x86 Thomas Gleixner <tglx@linutronix.de> - 2016-10-17 13:00 +0200
    [PATCH v4 07/18] x86/intel_rdt: Add Haswell feature discovery "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
      Re: [PATCH v4 07/18] x86/intel_rdt: Add Haswell feature discovery Thomas Gleixner <tglx@linutronix.de> - 2016-10-17 13:10 +0200

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

#1502083 — Re: [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters from CPUID

On Mon, Oct 17, 2016 at 03:45:32PM +0200, Thomas Gleixner wrote:
> On Fri, 14 Oct 2016, Fenghua Yu wrote:
> > +/**
> > + * struct rdt_resource - attributes of an RDT resource
> > + * @enabled:			Is this feature enabled on this machine
> > + * @name:			Name to use in "schemata" file
> > + * @max_closid:			Maximum number of CLOSIDs supported
> > + * @num_closid:			Current number of CLOSIDs available
> > + * @max_cbm:			Largest Cache Bit Mask allowed
> > + * @min_cbm_bits:		Minimum number of bits to be set in a cache
> 
> That should be 'number of consecutive bits', right?

Change to "Minimum number of consecutive bits to be set in a cache", is
that ok?

It's 2 on Haswell. It's 1 in other cases i.e. L3 on Broadwell and
Skylake servers etc.

> 
> > + *				bit mask
> > + * @domains:			All domains for this resource
> > + * @num_domains:		Number of domains active
> > + * @msr_base:			Base MSR address for CBMs
> > + * @cdp_capable:		Code/Data Prioritization available
> > + * @cdp_enabled:		Code/Data Prioritization enabled
> 
> I wonder whether this is the proper abstraction level. We might as well do
> the following:
> 
> rdtresources[] = {
>      {
> 	.name	= "L3",
>      },
>      {
> 	.name	= "L3Data",
>      },
>      {
> 	.name	= "L3Code",
>      },
> 
> and enable either L3 or L3Data+L3Code. Not sure if that makes things
> simpler, but it's definitely worth a thought or two.

This way will be better than having cdp_enabled/capable for L3 and not
for L2.  And this doesn't change current userinterface design either,
I think.

> 
> > +#define for_each_rdt_resource(r)	\
> > +	for (r = rdt_resources_all; r->name; r++) \
> > +		if (r->enabled)
> 
> So the resource array must be NULL terminated, right? You might as well use
> 
>    r < rdt_resources_all + ARRAY_SIZE(rdt_resources_all)
> 
> as the loop condition. So you avoid the NULL termination.

Will do.

> 
> > +
> > +#define IA32_L3_CBM_BASE	0xc90
> > +extern struct rdt_resource rdt_resources_all[];
> 
> Please visually split this. CBM_BASE has nothing to do with the resource
> array.
> 
> > +#define domain_init(name) LIST_HEAD_INIT(rdt_resources_all[name].domains)
> 
> name is really misleading here. Please use id and make this an inline
> function.
> 
> > +struct rdt_resource rdt_resources_all[] = {
> 
> >  static inline bool get_rdt_resources(void)
> >  {
> > +	struct rdt_resource *r;
> >  	bool ret = false;
> >  
> >  	if (boot_cpu_data.x86_vendor == X86_VENDOR_INTEL &&
> > @@ -74,20 +105,53 @@ static inline bool get_rdt_resources(void)
> >  
> >  	if (!boot_cpu_has(X86_FEATURE_RDT_A))
> >  		return false;
> > -	if (boot_cpu_has(X86_FEATURE_CAT_L3))
> > +	if (boot_cpu_has(X86_FEATURE_CAT_L3)) {
> > +		union cpuid_0x10_1_eax eax;
> > +		union cpuid_0x10_1_edx edx;
> > +		u32 ebx, ecx;
> > +
> > +		r = &rdt_resources_all[RDT_RESOURCE_L3];
> > +		cpuid_count(0x00000010, 1, &eax.full, &ebx, &ecx, &edx.full);
> > +		r->max_closid = edx.split.cos_max + 1;
> > +		r->num_closid = r->max_closid;
> > +		r->cbm_len = eax.split.cbm_len + 1;
> > +		r->max_cbm = BIT_MASK(eax.split.cbm_len + 1) - 1;
> > +		if (boot_cpu_has(X86_FEATURE_CDP_L3))
> > +			r->cdp_capable = true;
> > +		r->enabled = true;
> > +
> >  		ret = true;
> > +	}
> > +	if (boot_cpu_has(X86_FEATURE_CAT_L2)) {
> > +		union cpuid_0x10_1_eax eax;
> > +		union cpuid_0x10_1_edx edx;
> > +		u32 ebx, ecx;
> > +
> > +		/* CPUID 0x10.2 fields are same format at 0x10.1 */
> > +		r = &rdt_resources_all[RDT_RESOURCE_L2];
> > +		cpuid_count(0x00000010, 2, &eax.full, &ebx, &ecx, &edx.full);
> > +		r->max_closid = edx.split.cos_max + 1;
> > +		r->num_closid = r->max_closid;
> > +		r->cbm_len = eax.split.cbm_len + 1;
> > +		r->max_cbm = BIT_MASK(eax.split.cbm_len + 1) - 1;
> > +		r->enabled = true;
> 
> Copy and paste is a wonderful thing, right?
> 
> static void rdt_get_config(int idx, struct rdt_resource *r)
> {
> 	union cpuid_0x10_1_eax eax;
> 	union cpuid_0x10_1_edx edx;
> 	u32 ebx, ecx;
> 
> 	cpuid_count(0x00000010, idx, &eax.full, &ebx, &ecx, &edx.full);
> 	r->max_closid = edx.split.cos_max + 1;
> 	r->num_closid = r->max_closid;
> 	r->cbm_len = eax.split.cbm_len + 1;
> 	r->max_cbm = BIT_MASK(eax.split.cbm_len + 1) - 1;
> 	r->enabled = true;
> }
> 
> and and the call site:
> 
> 	if (boot_cpu_has(X86_FEATURE_CAT_L3)) {
> 		rdt_get_config(1, &rdt_resources_all[RDT_RESOURCE_L3]);
> 		if (boot_cpu_has(X86_FEATURE_CDP_L3))
> 			r->cdp_capable = true;
> 		ret = true;
> 	}
> 
> 	if (boot_cpu_has(X86_FEATURE_CAT_L2)) {
> 		rdt_get_config(2, &rdt_resources_all[RDT_RESOURCE_L2]);
> 		ret = true;
> 	}
> 
> Hmm?

Sure.

Thanks.

-Fenghua

[toc] | [prev] | [next] | [standalone]

#1502184 — Re: [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters from CPUID

> > I wonder whether this is the proper abstraction level. We might as well do
> > the following:
> > 
> > rdtresources[] = {
> >      {
> > 	.name	= "L3",
> >      },
> >      {
> > 	.name	= "L3Data",
> >      },
> >      {
> > 	.name	= "L3Code",
> >      },
> > 
> > and enable either L3 or L3Data+L3Code. Not sure if that makes things
> > simpler, but it's definitely worth a thought or two.
> 
> This way will be better than having cdp_enabled/capable for L3 and not
> for L2.  And this doesn't change current userinterface design either,
> I think.

User interface would change if you did this. The schemata file would
look like this with CDP enabled:

# cat schemata
L3Data:0=fffff;1=fffff;2=fffff;3=fffff
L3Code:0=fffff;1=fffff;2=fffff;3=fffff

but that is easier to read than the current:

# cat schemata
L3:0=fffff,fffff;1=fffff,fffff;2=fffff,fffff;3=fffff,fffff

which gives you no clue on which mask is code and which is data.

We'd also end up with "info/L3Data/" and "info/L3code/"
which would be a little redundant (since the files in each
would contain the same numbers), but perhaps that is worth
it to get the better schemata file.

-Tony

[toc] | [prev] | [next] | [standalone]

#1502199 — RE: [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters from CPUID

> > > I wonder whether this is the proper abstraction level. We might as
> > > well do the following:
> > >
> > > rdtresources[] = {
> > >      {
> > > 	.name	= "L3",
> > >      },
> > >      {
> > > 	.name	= "L3Data",
> > >      },
> > >      {
> > > 	.name	= "L3Code",
> > >      },
> > >
> > > and enable either L3 or L3Data+L3Code. Not sure if that makes things
> > > simpler, but it's definitely worth a thought or two.
> >
> > This way will be better than having cdp_enabled/capable for L3 and not
> > for L2.  And this doesn't change current userinterface design either,
> > I think.
> 
> User interface would change if you did this. The schemata file would look like
> this with CDP enabled:
> 
> # cat schemata
> L3Data:0=fffff;1=fffff;2=fffff;3=fffff
> L3Code:0=fffff;1=fffff;2=fffff;3=fffff
> 
> but that is easier to read than the current:
> 
> # cat schemata
> L3:0=fffff,fffff;1=fffff,fffff;2=fffff,fffff;3=fffff,fffff
> 
> which gives you no clue on which mask is code and which is data.

Right.

Also changing to uniform format <resname>:<id1>=cbm1;<id2>=cbm2;...
is lot easier to parse schemata line in CDP mode.

So I'll change the code and doc to have two new resources: L3Data and L3Code for CDP mode.

> 
> We'd also end up with "info/L3Data/" and "info/L3code/"
> which would be a little redundant (since the files in each would contain the
> same numbers), but perhaps that is worth it to get the better schemata file.
> 
> -Tony

[toc] | [prev] | [next] | [standalone]

#1502434 — Re: [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters from CPUID

On Mon, Oct 17, 2016 at 09:43:41AM -0700, Yu, Fenghua wrote:
> > > > I wonder whether this is the proper abstraction level. We might as
> > > > well do the following:
> > > >
> > > > rdtresources[] = {
> > > >      {
> > > > 	.name	= "L3",
> > > >      },
> > > >      {
> > > > 	.name	= "L3Data",
> > > >      },
> > > >      {
> > > > 	.name	= "L3Code",
> > > >      },
> > > >
> > > > and enable either L3 or L3Data+L3Code. Not sure if that makes things
> > > > simpler, but it's definitely worth a thought or two.
> > >
> > > This way will be better than having cdp_enabled/capable for L3 and not
> > > for L2.  And this doesn't change current userinterface design either,
> > > I think.
> > 
> > User interface would change if you did this. The schemata file would look like
> > this with CDP enabled:
> > 
> > # cat schemata
> > L3Data:0=fffff;1=fffff;2=fffff;3=fffff
> > L3Code:0=fffff;1=fffff;2=fffff;3=fffff
> > 
> > but that is easier to read than the current:
> > 
> > # cat schemata
> > L3:0=fffff,fffff;1=fffff,fffff;2=fffff,fffff;3=fffff,fffff
> > 
> > which gives you no clue on which mask is code and which is data.
> 
> Right.
> 
> Also changing to uniform format <resname>:<id1>=cbm1;<id2>=cbm2;...
> is lot easier to parse schemata line in CDP mode.
> 
> So I'll change the code and doc to have two new resources: L3Data and L3Code for CDP mode.

Doc change (fold into part 05):
diff --git a/Documentation/x86/intel_rdt_ui.txt b/Documentation/x86/intel_rdt_ui.txt
index e56781952f42..b9f634c9a058 100644
--- a/Documentation/x86/intel_rdt_ui.txt
+++ b/Documentation/x86/intel_rdt_ui.txt
@@ -97,13 +97,18 @@ With CDP disabled the L3 schemata format is:
 
 L3 details (CDP enabled via mount option to resctrl)
 ----------------------------------------------------
-When CDP is enabled, you need to specify separate cache bit masks for
-code and data access. The generic format is:
+When CDP is enabled L3 control is split into two separate resources
+so you can specify independent masks for code and data like this:
 
-	L3:<cache_id0>=<d_cbm>,<i_cbm>;<cache_id1>=<d_cbm>,<i_cbm>;...
+	L3data:<cache_id0>=<cbm>;<cache_id1>=<cbm>;...
+	L3code:<cache_id0>=<cbm>;<cache_id1>=<cbm>;...
 
-where the d_cbm masks are for data access, and the i_cbm masks for code.
+L2 details
+----------
+L2 cache does not support code and data prioritization, so the
+schemata format is always:
 
+	L2:<cache_id0>=<cbm>;<cache_id1>=<cbm>;...
 
 Example 1
 ---------

[toc] | [prev] | [next] | [standalone]

#1502258 — Re: [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters from CPUID

On Mon, 17 Oct 2016, Luck, Tony wrote:
> > > I wonder whether this is the proper abstraction level. We might as well do
> > > the following:
> > > 
> > > rdtresources[] = {
> > >      {
> > > 	.name	= "L3",
> > >      },
> > >      {
> > > 	.name	= "L3Data",
> > >      },
> > >      {
> > > 	.name	= "L3Code",
> > >      },
> > > 
> > > and enable either L3 or L3Data+L3Code. Not sure if that makes things
> > > simpler, but it's definitely worth a thought or two.
> > 
> > This way will be better than having cdp_enabled/capable for L3 and not
> > for L2.  And this doesn't change current userinterface design either,
> > I think.
> 
> User interface would change if you did this. The schemata file would
> look like this with CDP enabled:
> 
> # cat schemata
> L3Data:0=fffff;1=fffff;2=fffff;3=fffff
> L3Code:0=fffff;1=fffff;2=fffff;3=fffff
> 
> but that is easier to read than the current:
> 
> # cat schemata
> L3:0=fffff,fffff;1=fffff,fffff;2=fffff,fffff;3=fffff,fffff
> 
> which gives you no clue on which mask is code and which is data.

Indeed.
 
> We'd also end up with "info/L3Data/" and "info/L3code/"
> which would be a little redundant (since the files in each
> would contain the same numbers), but perhaps that is worth
> it to get the better schemata file.

I think so. Making the user interface more intuitive is always worth it.

Thanks,

	tglx

[toc] | [prev] | [next] | [standalone]

#1502250 — Re: [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters from CPUID

On Mon, 17 Oct 2016, Fenghua Yu wrote:
> On Mon, Oct 17, 2016 at 03:45:32PM +0200, Thomas Gleixner wrote:
> > On Fri, 14 Oct 2016, Fenghua Yu wrote:
> > > +/**
> > > + * struct rdt_resource - attributes of an RDT resource
> > > + * @enabled:			Is this feature enabled on this machine
> > > + * @name:			Name to use in "schemata" file
> > > + * @max_closid:			Maximum number of CLOSIDs supported
> > > + * @num_closid:			Current number of CLOSIDs available
> > > + * @max_cbm:			Largest Cache Bit Mask allowed
> > > + * @min_cbm_bits:		Minimum number of bits to be set in a cache
> > 
> > That should be 'number of consecutive bits', right?
> 
> Change to "Minimum number of consecutive bits to be set in a cache", is
> that ok?

Yes.
 
Thanks,

	tglx

[toc] | [prev] | [next] | [standalone]

#1502276 — Re: [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters from CPUID

On Mon, 17 Oct 2016, Fenghua Yu wrote:
> On Mon, Oct 17, 2016 at 03:45:32PM +0200, Thomas Gleixner wrote:
> > I wonder whether this is the proper abstraction level. We might as well do
> > the following:
> > 
> > rdtresources[] = {
> >      {
> > 	.name	= "L3",
> >      },
> >      {
> > 	.name	= "L3Data",
> >      },
> >      {
> > 	.name	= "L3Code",
> >      },
> > 
> > and enable either L3 or L3Data+L3Code. Not sure if that makes things
> > simpler, but it's definitely worth a thought or two.
> 
> This way will be better than having cdp_enabled/capable for L3 and not
> for L2.

So you need to change the struct to have capable and enabled

> > static void rdt_get_config(int idx, struct rdt_resource *r)
> > {
> > 	union cpuid_0x10_1_eax eax;
> > 	union cpuid_0x10_1_edx edx;
> > 	u32 ebx, ecx;
> > 
> > 	cpuid_count(0x00000010, idx, &eax.full, &ebx, &ecx, &edx.full);
> > 	r->max_closid = edx.split.cos_max + 1;
> > 	r->num_closid = r->max_closid;
> > 	r->cbm_len = eax.split.cbm_len + 1;
> > 	r->max_cbm = BIT_MASK(eax.split.cbm_len + 1) - 1;
> > 	r->enabled = true;

And set 

    	r->capable = true; 

here instead of r->enabled and set r->enabled at mount time for the
resources depending on the mount flags.

Thanks,

	tglx


	

[toc] | [prev] | [next] | [standalone]

#1502323 — Re: [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters from CPUID

On Mon, Oct 17, 2016 at 07:02:19PM +0200, Thomas Gleixner wrote:
> On Mon, 17 Oct 2016, Fenghua Yu wrote:
> > On Mon, Oct 17, 2016 at 03:45:32PM +0200, Thomas Gleixner wrote:
> > > I wonder whether this is the proper abstraction level. We might as well do
> > > the following:
> > > 
> > > rdtresources[] = {
> > >      {
> > > 	.name	= "L3",
> > >      },
> > >      {
> > > 	.name	= "L3Data",
> > >      },
> > >      {
> > > 	.name	= "L3Code",
> > >      },
> > > 
> > > and enable either L3 or L3Data+L3Code. Not sure if that makes things
> > > simpler, but it's definitely worth a thought or two.
> > 
> > This way will be better than having cdp_enabled/capable for L3 and not
> > for L2.
> 
> So you need to change the struct to have capable and enabled
> 
> > > static void rdt_get_config(int idx, struct rdt_resource *r)
> > > {
> > > 	union cpuid_0x10_1_eax eax;
> > > 	union cpuid_0x10_1_edx edx;
> > > 	u32 ebx, ecx;
> > > 
> > > 	cpuid_count(0x00000010, idx, &eax.full, &ebx, &ecx, &edx.full);
> > > 	r->max_closid = edx.split.cos_max + 1;
> > > 	r->num_closid = r->max_closid;
> > > 	r->cbm_len = eax.split.cbm_len + 1;
> > > 	r->max_cbm = BIT_MASK(eax.split.cbm_len + 1) - 1;
> > > 	r->enabled = true;
> 
> And set 
> 
>     	r->capable = true; 
> 
> here instead of r->enabled and set r->enabled at mount time for the
> resources depending on the mount flags.

Neat code change!
 
Thanks,

-Fenghua

[toc] | [prev] | [next] | [standalone]

#1501219 — [PATCH v4 01/18] Documentation, ABI: Add a document entry for cache id

From: Fenghua Yu <fenghua.yu@intel.com>

Add an ABI document entry for /sys/devices/system/cpu/cpu*/cache/index*/id.

Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Signed-off-by: Tony Luck <tony.luck@intel.com>
---
 Documentation/ABI/testing/sysfs-devices-system-cpu | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/Documentation/ABI/testing/sysfs-devices-system-cpu b/Documentation/ABI/testing/sysfs-devices-system-cpu
index 4987417..b1c3d69 100644
--- a/Documentation/ABI/testing/sysfs-devices-system-cpu
+++ b/Documentation/ABI/testing/sysfs-devices-system-cpu
@@ -272,6 +272,22 @@ Description:	Parameters for the CPU cache attributes
 				     the modified cache line is written to main
 				     memory only when it is replaced
 
+
+What:		/sys/devices/system/cpu/cpu*/cache/index*/id
+Date:		September 2016
+Contact:	Linux kernel mailing list <linux-kernel@vger.kernel.org>
+Description:	Cache id
+
+		The id provides a unique name for a specific instance of
+		a cache of a particular type. E.g. there may be a level
+		3 unified cache on each socket in a server and we may
+		assign them ids 0, 1, 2, ...
+
+		Note that id value may not be contiguous. E.g. level 1
+		caches typically exist per core, but there may not be a
+		power of two cores on a socket, so these caches may be
+		numbered 0, 1, 2, 3, 4, 5, 8, 9, 10, ...
+
 What:		/sys/devices/system/cpu/cpuX/cpufreq/throttle_stats
 		/sys/devices/system/cpu/cpuX/cpufreq/throttle_stats/turbo_stat
 		/sys/devices/system/cpu/cpuX/cpufreq/throttle_stats/sub_turbo_stat
-- 
2.5.0

[toc] | [prev] | [next] | [standalone]

#1501861 — Re: [PATCH v4 01/18] Documentation, ABI: Add a document entry for cache id

On Fri, 14 Oct 2016, Fenghua Yu wrote:

> From: Fenghua Yu <fenghua.yu@intel.com>
> 
> Add an ABI document entry for /sys/devices/system/cpu/cpu*/cache/index*/id.
> 
> Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
> Signed-off-by: Tony Luck <tony.luck@intel.com>

The SOB chain here is bogus ....

> ---
>  Documentation/ABI/testing/sysfs-devices-system-cpu | 16 ++++++++++++++++
>  1 file changed, 16 insertions(+)
> 
> diff --git a/Documentation/ABI/testing/sysfs-devices-system-cpu b/Documentation/ABI/testing/sysfs-devices-system-cpu
> index 4987417..b1c3d69 100644
> --- a/Documentation/ABI/testing/sysfs-devices-system-cpu
> +++ b/Documentation/ABI/testing/sysfs-devices-system-cpu
> @@ -272,6 +272,22 @@ Description:	Parameters for the CPU cache attributes
>  				     the modified cache line is written to main
>  				     memory only when it is replaced
>  
> +
> +What:		/sys/devices/system/cpu/cpu*/cache/index*/id
> +Date:		September 2016
> +Contact:	Linux kernel mailing list <linux-kernel@vger.kernel.org>
> +Description:	Cache id
> +
> +		The id provides a unique name for a specific instance of

s/name/number/

> +		a cache of a particular type. E.g. there may be a level
> +		3 unified cache on each socket in a server and we may
> +		assign them ids 0, 1, 2, ...
> +
> +		Note that id value may not be contiguous. E.g. level 1

Note, that the id values can be non-contiguous.

> +		caches typically exist per core, but there may not be a
> +		power of two cores on a socket, so these caches may be
> +		numbered 0, 1, 2, 3, 4, 5, 8, 9, 10, ...
> +

Thanks,

	tglx

[toc] | [prev] | [next] | [standalone]

#1501220 — [PATCH v4 18/18] MAINTAINERS: Add maintainer for Intel RDT resource allocation

From: Fenghua Yu <fenghua.yu@intel.com>

We create five new files for Intel RDT resource allocation:
arch/x86/kernel/cpu/intel_rdt.c
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
arch/x86/kernel/cpu/intel_rdt_schemata.c
arch/x86/include/asm/intel_rdt.h
Documentation/x86/intel_rdt_ui.txt

Fenghua Yu will maintain this code.

Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Signed-off-by: Tony Luck <tony.luck@intel.com>
---
 MAINTAINERS | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index f593300..bcb75e9 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -9845,6 +9845,14 @@ L:	linux-rdma@vger.kernel.org
 S:	Supported
 F:	drivers/infiniband/sw/rdmavt
 
+RDT - RESOURCE ALLOCATION
+M:	Fenghua Yu <fenghua.yu@intel.com>
+L:	linux-kernel@vger.kernel.org
+S:	Supported
+F:	arch/x86/kernel/cpu/intel_rdt*
+F:	arch/x86/include/asm/intel_rdt*
+F:	Documentation/x86/intel_rdt*
+
 READ-COPY UPDATE (RCU)
 M:	"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
 M:	Josh Triplett <josh@joshtriplett.org>
-- 
2.5.0

[toc] | [prev] | [next] | [standalone]

#1501221 — [PATCH v4 04/18] x86/intel_rdt: Feature discovery

From: Fenghua Yu <fenghua.yu@intel.com>

Check CPUID leaves for all the Resource Director Technology (RDT)
Cache Allocation Technology (CAT) bits.

Presence of allocation features:
  CPUID.(EAX=7H, ECX=0):EBX[bit 15]	X86_FEATURE_RDT_A

L2 and L3 caches are each separately enabled:
  CPUID.(EAX=10H, ECX=0):EBX[bit 1]	X86_FEATURE_CAT_L3
  CPUID.(EAX=10H, ECX=0):EBX[bit 2]	X86_FEATURE_CAT_L2

L3 cache may support independent control of allocation for
code and data (CDP = Code/Data Prioritization):
  CPUID.(EAX=10H, ECX=1):ECX[bit 2]	X86_FEATURE_CDP_L3

Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Signed-off-by: Tony Luck <tony.luck@intel.com>
---
 arch/x86/include/asm/cpufeatures.h | 5 +++++
 arch/x86/kernel/cpu/scattered.c    | 3 +++
 2 files changed, 8 insertions(+)

diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index 92a8308..64dd8274 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -196,6 +196,10 @@
 
 #define X86_FEATURE_INTEL_PT	( 7*32+15) /* Intel Processor Trace */
 
+#define X86_FEATURE_CAT_L3	( 7*32+16) /* Cache Allocation Technology L3 */
+#define X86_FEATURE_CAT_L2	( 7*32+17) /* Cache Allocation Technology L2 */
+#define X86_FEATURE_CDP_L3	( 7*32+18) /* Code and Data Prioritization L3 */
+
 /* Virtualization flags: Linux defined, word 8 */
 #define X86_FEATURE_TPR_SHADOW  ( 8*32+ 0) /* Intel TPR Shadow */
 #define X86_FEATURE_VNMI        ( 8*32+ 1) /* Intel Virtual NMI */
@@ -220,6 +224,7 @@
 #define X86_FEATURE_RTM		( 9*32+11) /* Restricted Transactional Memory */
 #define X86_FEATURE_CQM		( 9*32+12) /* Cache QoS Monitoring */
 #define X86_FEATURE_MPX		( 9*32+14) /* Memory Protection Extension */
+#define X86_FEATURE_RDT_A	( 9*32+15) /* Resource Director Technology Allocation */
 #define X86_FEATURE_AVX512F	( 9*32+16) /* AVX-512 Foundation */
 #define X86_FEATURE_AVX512DQ	( 9*32+17) /* AVX-512 DQ (Double/Quad granular) Instructions */
 #define X86_FEATURE_RDSEED	( 9*32+18) /* The RDSEED instruction */
diff --git a/arch/x86/kernel/cpu/scattered.c b/arch/x86/kernel/cpu/scattered.c
index 8cb57df..11f39a2 100644
--- a/arch/x86/kernel/cpu/scattered.c
+++ b/arch/x86/kernel/cpu/scattered.c
@@ -34,6 +34,9 @@ void init_scattered_cpuid_features(struct cpuinfo_x86 *c)
 		{ X86_FEATURE_INTEL_PT,		CR_EBX,25, 0x00000007, 0 },
 		{ X86_FEATURE_APERFMPERF,	CR_ECX, 0, 0x00000006, 0 },
 		{ X86_FEATURE_EPB,		CR_ECX, 3, 0x00000006, 0 },
+		{ X86_FEATURE_CAT_L3,		CR_EBX, 1, 0x00000010, 0 },
+		{ X86_FEATURE_CAT_L2,		CR_EBX, 2, 0x00000010, 0 },
+		{ X86_FEATURE_CDP_L3,		CR_ECX, 2, 0x00000010, 1 },
 		{ X86_FEATURE_HW_PSTATE,	CR_EDX, 7, 0x80000007, 0 },
 		{ X86_FEATURE_CPB,		CR_EDX, 9, 0x80000007, 0 },
 		{ X86_FEATURE_PROC_FEEDBACK,	CR_EDX,11, 0x80000007, 0 },
-- 
2.5.0

[toc] | [prev] | [next] | [standalone]

#1501222 — [PATCH v4 11/18] x86/intel_rdt: Add basic resctrl filesystem support

From: Fenghua Yu <fenghua.yu@intel.com>

Use kernfs as basis for our user interface filesystem. This patch
supports mount/umount, and one mount parameter "cdp" to enable code/data
prioritization (though all we do at this point is ensure that the system
can support CDP).  The file system is not populated yet in this patch.

Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Signed-off-by: Tony Luck <tony.luck@intel.com>
---
 arch/x86/include/asm/intel_rdt.h         |  22 +++
 arch/x86/kernel/cpu/Makefile             |   2 +-
 arch/x86/kernel/cpu/intel_rdt.c          |   8 +-
 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 235 +++++++++++++++++++++++++++++++
 include/uapi/linux/magic.h               |   1 +
 5 files changed, 266 insertions(+), 2 deletions(-)
 create mode 100644 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c

diff --git a/arch/x86/include/asm/intel_rdt.h b/arch/x86/include/asm/intel_rdt.h
index b3df691..f7acae2 100644
--- a/arch/x86/include/asm/intel_rdt.h
+++ b/arch/x86/include/asm/intel_rdt.h
@@ -5,6 +5,23 @@
 #define IA32_L2_CBM_BASE	0xd10
 
 /**
+ * struct rdtgroup - store rdtgroup's data in resctrl file system.
+ * @kn:				kernfs node
+ * @rdtgroup_list:		linked list for all rdtgroups
+ * @closid:			closid for this rdtgroup
+ */
+struct rdtgroup {
+	struct kernfs_node	*kn;
+	struct list_head	rdtgroup_list;
+	int			closid;
+};
+
+/* List of all resource groups */
+extern struct list_head rdt_all_groups;
+
+int __init rdtgroup_init(void);
+
+/**
  * struct rdt_resource - attributes of an RDT resource
  * @enabled:			Is this feature enabled on this machine
  * @name:			Name to use in "schemata" file
@@ -42,6 +59,7 @@ struct rdt_resource {
 	for (r = rdt_resources_all; r->name; r++) \
 		if (r->enabled)
 
+#define IA32_L3_QOS_CFG		0xc81
 #define IA32_L3_CBM_BASE	0xc90
 
 /**
@@ -73,6 +91,10 @@ struct msr_param {
 extern struct mutex rdtgroup_mutex;
 
 extern struct rdt_resource rdt_resources_all[];
+extern struct rdtgroup rdtgroup_default;
+extern struct static_key_false rdt_enable_key;
+
+int __init rdtgroup_init(void);
 
 enum {
 	RDT_RESOURCE_L3,
diff --git a/arch/x86/kernel/cpu/Makefile b/arch/x86/kernel/cpu/Makefile
index cf4bfd0..b4334e8 100644
--- a/arch/x86/kernel/cpu/Makefile
+++ b/arch/x86/kernel/cpu/Makefile
@@ -34,7 +34,7 @@ obj-$(CONFIG_CPU_SUP_CENTAUR)		+= centaur.o
 obj-$(CONFIG_CPU_SUP_TRANSMETA_32)	+= transmeta.o
 obj-$(CONFIG_CPU_SUP_UMC_32)		+= umc.o
 
-obj-$(CONFIG_INTEL_RDT_A)	+= intel_rdt.o
+obj-$(CONFIG_INTEL_RDT_A)	+= intel_rdt.o intel_rdt_rdtgroup.o
 
 obj-$(CONFIG_X86_MCE)			+= mcheck/
 obj-$(CONFIG_MTRR)			+= mtrr/
diff --git a/arch/x86/kernel/cpu/intel_rdt.c b/arch/x86/kernel/cpu/intel_rdt.c
index 0b47ea9..3d4a125 100644
--- a/arch/x86/kernel/cpu/intel_rdt.c
+++ b/arch/x86/kernel/cpu/intel_rdt.c
@@ -266,7 +266,7 @@ static int intel_rdt_offline_cpu(unsigned int cpu)
 static int __init intel_rdt_late_init(void)
 {
 	struct rdt_resource *r;
-	int state;
+	int state, ret;
 
 	if (!get_rdt_resources())
 		return -ENODEV;
@@ -277,6 +277,12 @@ static int __init intel_rdt_late_init(void)
 	if (state < 0)
 		return state;
 
+	ret = rdtgroup_init();
+	if (ret) {
+		cpuhp_remove_state(state);
+		return ret;
+	}
+
 	for_each_rdt_resource(r)
 		pr_info("Intel RDT %s allocation %s detected\n", r->name,
 			r->cdp_capable ? " (with CDP)" : "");
diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
new file mode 100644
index 0000000..b2bbd04
--- /dev/null
+++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
@@ -0,0 +1,235 @@
+/*
+ * User interface for Resource Alloction in Resource Director Technology(RDT)
+ *
+ * Copyright (C) 2016 Intel Corporation
+ *
+ * Author: Fenghua Yu <fenghua.yu@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * More information about RDT be found in the Intel (R) x86 Architecture
+ * Software Developer Manual.
+ */
+
+#define pr_fmt(fmt)	KBUILD_MODNAME ": " fmt
+
+#include <linux/fs.h>
+#include <linux/sysfs.h>
+#include <linux/kernfs.h>
+#include <linux/slab.h>
+
+#include <uapi/linux/magic.h>
+
+#include <asm/intel_rdt.h>
+
+DEFINE_STATIC_KEY_FALSE(rdt_enable_key);
+struct kernfs_root *rdt_root;
+struct rdtgroup rdtgroup_default;
+LIST_HEAD(rdt_all_groups);
+
+static void l3_qos_cfg_update(void *arg)
+{
+	struct rdt_resource *r = arg;
+
+	wrmsrl(IA32_L3_QOS_CFG, r->cdp_enabled);
+}
+
+static void set_l3_qos_cfg(struct rdt_resource *r)
+{
+	struct list_head *l;
+	struct rdt_domain *d;
+	struct cpumask cpu_mask;
+	int cpu;
+
+	cpumask_clear(&cpu_mask);
+	list_for_each(l, &r->domains) {
+		d = list_entry(l, struct rdt_domain, list);
+		cpumask_set_cpu(cpumask_any(&d->cpu_mask), &cpu_mask);
+	}
+	cpu = get_cpu();
+	/* Update QOS_CFG MSR on this cpu if it's in cpu_mask. */
+	if (cpumask_test_cpu(cpu, &cpu_mask))
+		l3_qos_cfg_update(r);
+	/* Update QOS_CFG MSR on all other cpus in cpu_mask. */
+	smp_call_function_many(&cpu_mask, l3_qos_cfg_update, r, 1);
+	put_cpu();
+}
+
+static int parse_rdtgroupfs_options(char *data)
+{
+	char *token, *o = data;
+	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3];
+
+	r->cdp_enabled = false;
+	while ((token = strsep(&o, ",")) != NULL) {
+		if (!*token)
+			return -EINVAL;
+
+		if (!strcmp(token, "cdp"))
+			if (r->enabled && r->cdp_capable)
+				r->cdp_enabled = true;
+	}
+
+	return 0;
+}
+
+static struct dentry *rdt_mount(struct file_system_type *fs_type,
+				int flags, const char *unused_dev_name,
+				void *data)
+{
+	struct dentry *dentry;
+	int ret;
+	bool new_sb;
+
+	mutex_lock(&rdtgroup_mutex);
+	/*
+	 * resctrl file system can only be mounted once.
+	 */
+	if (static_branch_unlikely(&rdt_enable_key)) {
+		dentry = ERR_PTR(-EBUSY);
+		goto out;
+	}
+
+	ret = parse_rdtgroupfs_options(data);
+	if (ret) {
+		dentry = ERR_PTR(ret);
+		goto out;
+	}
+
+	dentry = kernfs_mount(fs_type, flags, rdt_root,
+			      RDTGROUP_SUPER_MAGIC, &new_sb);
+	if (IS_ERR(dentry))
+		goto out;
+	if (!new_sb) {
+		dentry = ERR_PTR(-EINVAL);
+		goto out;
+	}
+	if (rdt_resources_all[RDT_RESOURCE_L3].cdp_capable)
+		set_l3_qos_cfg(&rdt_resources_all[RDT_RESOURCE_L3]);
+	static_branch_enable(&rdt_enable_key);
+
+out:
+	mutex_unlock(&rdtgroup_mutex);
+
+	return dentry;
+}
+
+static void reset_all_cbms(struct rdt_resource *r)
+{
+	struct list_head *l;
+	struct rdt_domain *d;
+	struct msr_param msr_param;
+	struct cpumask cpu_mask;
+	int i, cpu;
+
+	cpumask_clear(&cpu_mask);
+	msr_param.res = r;
+	msr_param.low = 0;
+	msr_param.high = r->max_closid;
+
+	list_for_each(l, &r->domains) {
+		d = list_entry(l, struct rdt_domain, list);
+		cpumask_set_cpu(cpumask_any(&d->cpu_mask), &cpu_mask);
+
+		for (i = 0; i < r->max_closid; i++)
+			d->cbm[i] = r->max_cbm;
+	}
+	cpu = get_cpu();
+	/* Update CBM on this cpu if it's in cpu_mask. */
+	if (cpumask_test_cpu(cpu, &cpu_mask))
+		rdt_cbm_update(&msr_param);
+	/* Updte CBM on all other cpus in cpu_mask. */
+	smp_call_function_many(&cpu_mask, rdt_cbm_update, &msr_param, 1);
+	put_cpu();
+}
+
+static void rdt_kill_sb(struct super_block *sb)
+{
+	struct rdt_resource *r;
+
+	mutex_lock(&rdtgroup_mutex);
+
+	/*Put everything back to default values. */
+	for_each_rdt_resource(r)
+		reset_all_cbms(r);
+	r = &rdt_resources_all[RDT_RESOURCE_L3];
+	if (r->cdp_capable) {
+		r->cdp_enabled = 0;
+		set_l3_qos_cfg(r);
+	}
+
+	static_branch_disable(&rdt_enable_key);
+	kernfs_kill_sb(sb);
+	mutex_unlock(&rdtgroup_mutex);
+}
+
+static struct file_system_type rdt_fs_type = {
+	.name    = "resctrl",
+	.mount   = rdt_mount,
+	.kill_sb = rdt_kill_sb,
+};
+
+static struct kernfs_syscall_ops rdtgroup_kf_syscall_ops = {
+};
+
+static int __init rdtgroup_setup_root(void)
+{
+	rdt_root = kernfs_create_root(&rdtgroup_kf_syscall_ops,
+				      KERNFS_ROOT_CREATE_DEACTIVATED,
+				      &rdtgroup_default);
+	if (IS_ERR(rdt_root))
+		return PTR_ERR(rdt_root);
+
+	mutex_lock(&rdtgroup_mutex);
+
+	rdtgroup_default.closid = 0;
+	list_add(&rdtgroup_default.rdtgroup_list, &rdt_all_groups);
+
+	rdtgroup_default.kn = rdt_root->kn;
+	kernfs_activate(rdtgroup_default.kn);
+
+	mutex_unlock(&rdtgroup_mutex);
+
+	return 0;
+}
+
+/*
+ * rdtgroup_init - rdtgroup initialization
+ *
+ * Setup resctrl file system including set up root, create mount point,
+ * register rdtgroup filesystem, and initialize files under root directory.
+ *
+ * Return: 0 on success or -errno
+ */
+int __init rdtgroup_init(void)
+{
+	int ret = 0;
+
+	ret = rdtgroup_setup_root();
+	if (ret)
+		return ret;
+
+	ret = sysfs_create_mount_point(fs_kobj, "resctrl");
+	if (ret)
+		goto cleanup_root;
+
+	ret = register_filesystem(&rdt_fs_type);
+	if (ret)
+		goto cleanup_mountpoint;
+
+	return 0;
+
+cleanup_mountpoint:
+	sysfs_remove_mount_point(fs_kobj, "resctrl");
+cleanup_root:
+	kernfs_destroy_root(rdt_root);
+
+	return ret;
+}
diff --git a/include/uapi/linux/magic.h b/include/uapi/linux/magic.h
index e398bea..27ef03d 100644
--- a/include/uapi/linux/magic.h
+++ b/include/uapi/linux/magic.h
@@ -57,6 +57,7 @@
 #define CGROUP_SUPER_MAGIC	0x27e0eb
 #define CGROUP2_SUPER_MAGIC	0x63677270
 
+#define RDTGROUP_SUPER_MAGIC	0x7655821
 
 #define STACK_END_MAGIC		0x57AC6E9D
 
-- 
2.5.0

[toc] | [prev] | [next] | [standalone]

#1502372 — Re: [PATCH v4 11/18] x86/intel_rdt: Add basic resctrl filesystem support

On Fri, 14 Oct 2016, Fenghua Yu wrote:
> +++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
> + *
> + * More information about RDT be found in the Intel (R) x86 Architecture
> + * Software Developer Manual.

Yes, that's how it should look like.

> +static void l3_qos_cfg_update(void *arg)
> +{
> +	struct rdt_resource *r = arg;
> +
> +	wrmsrl(IA32_L3_QOS_CFG, r->cdp_enabled);
> +}
> +
> +static void set_l3_qos_cfg(struct rdt_resource *r)
> +{
> +	struct list_head *l;
> +	struct rdt_domain *d;
> +	struct cpumask cpu_mask;

You cannot have cpumasks on stack.

        cpumask_var_t mask;

        if (!zalloc_cpumask_var(&mask, GFP_KERNEL))
                return -ENOMEM;


> +	int cpu;
> +
> +	cpumask_clear(&cpu_mask);

That can go away then

> +	list_for_each(l, &r->domains) {

list_for_each_entry() again

> +		d = list_entry(l, struct rdt_domain, list);
> +		cpumask_set_cpu(cpumask_any(&d->cpu_mask), &cpu_mask);

A comment to explain what this does would be helpful.

> +	}
> +	cpu = get_cpu();
> +	/* Update QOS_CFG MSR on this cpu if it's in cpu_mask. */
> +	if (cpumask_test_cpu(cpu, &cpu_mask))
> +		l3_qos_cfg_update(r);
> +	/* Update QOS_CFG MSR on all other cpus in cpu_mask. */
> +	smp_call_function_many(&cpu_mask, l3_qos_cfg_update, r, 1);
> +	put_cpu();
> +}
> +
> +static int parse_rdtgroupfs_options(char *data)
> +{
> +	char *token, *o = data;
> +	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3];
> +
> +	r->cdp_enabled = false;
> +	while ((token = strsep(&o, ",")) != NULL) {
> +		if (!*token)
> +			return -EINVAL;
> +
> +		if (!strcmp(token, "cdp"))
> +			if (r->enabled && r->cdp_capable)
> +				r->cdp_enabled = true;
> +	}
> +
> +	return 0;
> +}
> +
> +static struct dentry *rdt_mount(struct file_system_type *fs_type,
> +				int flags, const char *unused_dev_name,
> +				void *data)
> +{
> +	struct dentry *dentry;
> +	int ret;
> +	bool new_sb;
> +
> +	mutex_lock(&rdtgroup_mutex);
> +	/*
> +	 * resctrl file system can only be mounted once.
> +	 */
> +	if (static_branch_unlikely(&rdt_enable_key)) {
> +		dentry = ERR_PTR(-EBUSY);
> +		goto out;
> +	}
> +
> +	ret = parse_rdtgroupfs_options(data);
> +	if (ret) {
> +		dentry = ERR_PTR(ret);
> +		goto out;
> +	}
> +
> +	dentry = kernfs_mount(fs_type, flags, rdt_root,
> +			      RDTGROUP_SUPER_MAGIC, &new_sb);

&new_sb is pointless here. It just tells the caller that a new superblock
has been created, So in case of a valid dentry new_sb will always be true,
and if anything failed in kernfs_mount() including the allocation of a new
superblock then new_sb is completely irrelevant as IS_ERR(dentry) will be
true. So you can just hand in NULL because you do not allow multiple
mounts.

> +	if (IS_ERR(dentry))
> +		goto out;
> +	if (!new_sb) {
> +		dentry = ERR_PTR(-EINVAL);
> +		goto out;
> +	}
> +	if (rdt_resources_all[RDT_RESOURCE_L3].cdp_capable)
> +		set_l3_qos_cfg(&rdt_resources_all[RDT_RESOURCE_L3]);
> +	static_branch_enable(&rdt_enable_key);
> +
> +out:
> +	mutex_unlock(&rdtgroup_mutex);
> +
> +	return dentry;
> +}
> +
> +static void reset_all_cbms(struct rdt_resource *r)
> +{
> +	struct list_head *l;
> +	struct rdt_domain *d;
> +	struct msr_param msr_param;
> +	struct cpumask cpu_mask;
> +	int i, cpu;
> +
> +	cpumask_clear(&cpu_mask);
> +	msr_param.res = r;
> +	msr_param.low = 0;
> +	msr_param.high = r->max_closid;
> +
> +	list_for_each(l, &r->domains) {
> +		d = list_entry(l, struct rdt_domain, list);

list_for_each_entry()

> +		cpumask_set_cpu(cpumask_any(&d->cpu_mask), &cpu_mask);
> +
> +		for (i = 0; i < r->max_closid; i++)
> +			d->cbm[i] = r->max_cbm;
> +	}
> +	cpu = get_cpu();
> +	/* Update CBM on this cpu if it's in cpu_mask. */
> +	if (cpumask_test_cpu(cpu, &cpu_mask))
> +		rdt_cbm_update(&msr_param);
> +	/* Updte CBM on all other cpus in cpu_mask. */

Update

> +	smp_call_function_many(&cpu_mask, rdt_cbm_update, &msr_param, 1);
> +	put_cpu();
> +}
> +

Thanks,

	tglx

[toc] | [prev] | [next] | [standalone]

#1501223 — [PATCH v4 05/18] Documentation, x86: Documentation for Intel resource allocation user interface

From: Fenghua Yu <fenghua.yu@intel.com>

The documentation describes user interface of how to allocate resource
in Intel RDT.

Please note that the documentation covers generic user interface. Current
patch set code only implemente CAT L3. CAT L2 code will be sent later.

Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Signed-off-by: Tony Luck <tony.luck@intel.com>
---
 Documentation/x86/intel_rdt_ui.txt | 162 +++++++++++++++++++++++++++++++++++++
 1 file changed, 162 insertions(+)
 create mode 100644 Documentation/x86/intel_rdt_ui.txt

diff --git a/Documentation/x86/intel_rdt_ui.txt b/Documentation/x86/intel_rdt_ui.txt
new file mode 100644
index 0000000..e567819
--- /dev/null
+++ b/Documentation/x86/intel_rdt_ui.txt
@@ -0,0 +1,162 @@
+User Interface for Resource Allocation in Intel Resource Director Technology
+
+Copyright (C) 2016 Intel Corporation
+
+Fenghua Yu <fenghua.yu@intel.com>
+Tony Luck <tony.luck@intel.com>
+
+This feature is enabled by the CONFIG_INTEL_RDT_A Kconfig and the
+X86 /proc/cpuinfo flag bits "rdt", "cat_l3" and "cdp_l3".
+
+To use the feature mount the file system:
+
+ # mount -t resctrl resctrl [-o cdp] /sys/fs/resctrl
+
+mount options are:
+
+"cdp": Enable code/data prioritization in L3 cache allocations.
+
+
+Resource groups
+---------------
+Resource groups are represented as directories in the resctrl file
+system. The default group is the root directory. Other groups may be
+created as desired by the system administrator using the "mkdir(1)"
+command, and removed using "rmdir(1)".
+
+There are three files associated with each group:
+
+"tasks": A list of tasks that belongs to this group. Tasks can be
+	added to a group by writing the task ID to the "tasks" file
+	(which will automatically remove them from the previous
+	group to which they belonged). New tasks created by fork(2)
+	and clone(2) are added to the same group as their parent.
+	If a pid is not in any sub partition, it is in root partition
+	(i.e. default partition).
+
+"cpus": A bitmask of logical CPUs assigned to this group. Writing
+	a new mask can add/remove CPUs from this group. Added CPUs
+	are removed from their previous group. Removed ones are
+	given to the default (root) group. You cannot remove CPUs
+	from the default group.
+
+"schemata": A list of all the resources available to this group.
+	Each resource has its own line and format - see below for
+	details.
+
+When a task is running the following rules define which resources
+are available to it:
+
+1) If the task is a member of a non-default group, then the schemata
+for that group is used.
+
+2) Else if the task belongs to the default group, but is running on a
+CPU that is assigned to some specific group, then the schemata for
+the CPU's group is used.
+
+3) Otherwise the schemata for the default group is used.
+
+
+Schemata files - general concepts
+---------------------------------
+Each line in the file describes one resource. The line starts with
+the name of the resource, followed by specific values to be applied
+in each of the instances of that resource on the system.
+
+Cache IDs
+---------
+On current generation systems there is one L3 cache per socket and L2
+caches are generally just shared by the hyperthreads on a core, but this
+isn't an architectural requirement. We could have multiple separate L3
+caches on a socket, multiple cores could share an L2 cache. So instead
+of using "socket" or "core" to define the set of logical cpus sharing
+a resource we use a "Cache ID". At a given cache level this will be a
+unique number across the whole system (but it isn't guaranteed to be a
+contiguous sequence, there may be gaps).  To find the ID for each logical
+CPU look in /sys/devices/system/cpu/cpu*/cache/index*/id
+
+Cache Bit Masks (CBM)
+---------------------
+For cache resources we describe the portion of the cache that is available
+for allocation using a bitmask. The maximum value of the mask is defined
+by each cpu model (and may be different for different cache levels). It
+is found using CPUID, but is also provided in the "info" directory of
+the resctrl file system in "info/{resource}/max_cbm_val". X86 hardware
+requires that these masks have all the '1' bits in a contiguous block. So
+0x3, 0x6 and 0xC are legal 4-bit masks with two bits set, but 0x5, 0x9
+and 0xA are not.  On a system with a 20-bit mask each bit represents 5%
+of the capacity of the cache. You could partition the cache into four
+equal parts with masks: 0x1f, 0x3e0, 0x7c00, 0xf8000.
+
+
+L3 details (code and data prioritization disabled)
+--------------------------------------------------
+With CDP disabled the L3 schemata format is:
+
+	L3:<cache_id0>=<cbm>;<cache_id1>=<cbm>;...
+
+L3 details (CDP enabled via mount option to resctrl)
+----------------------------------------------------
+When CDP is enabled, you need to specify separate cache bit masks for
+code and data access. The generic format is:
+
+	L3:<cache_id0>=<d_cbm>,<i_cbm>;<cache_id1>=<d_cbm>,<i_cbm>;...
+
+where the d_cbm masks are for data access, and the i_cbm masks for code.
+
+
+Example 1
+---------
+On a two socket machine (one L3 cache per socket) with just four bits
+for cache bit masks
+
+# mount -t resctrl resctrl /sys/fs/resctrl
+# cd /sys/fs/resctrl
+# mkdir p0 p1
+# echo "L3:0=3;1=c" > /sys/fs/resctrl/p0/schemata
+# echo "L3:0=3;1=3" > /sys/fs/resctrl/p1/schemata
+
+The default resource group is unmodified, so we have access to all parts
+of all caches (its schemata file reads "L3:0=f;1=f").
+
+Tasks that are under the control of group "p0" may only allocate from the
+"lower" 50% on cache ID 0, and the "upper" 50% of cache ID 1.
+Tasks in group "p1" use the "lower" 50% of cache on both sockets.
+
+Example 2
+---------
+Again two sockets, but this time with a more realistic 20-bit mask.
+
+Two real time tasks pid=1234 running on processor 0 and pid=5678 running on
+processor 1 on socket 0 on a 2-socket and dual core machine. To avoid noisy
+neighbors, each of the two real-time tasks exclusively occupies one quarter
+of L3 cache on socket 0.
+
+# mount -t resctrl resctrl /sys/fs/resctrl
+# cd /sys/fs/resctrl
+
+First we reset the schemata for the default group so that the "upper"
+50% of the L3 cache on socket 0 cannot be used by ordinary tasks:
+
+# echo "L3:0=3ff;1=fffff" > schemata
+
+Next we make a resource group for our first real time task and give
+it access to the "top" 25% of the cache on socket 0.
+
+# mkdir p0
+# echo "L3:0=f8000;1=fffff" > p0/schemata
+
+Finally we move our first real time task into this resource group. We
+also use taskset(1) to ensure the task always runs on a dedicated CPU
+on socket 0. Most uses of resource groups will also constrain which
+processors tasks run on.
+
+# echo 1234 > p0/tasks
+# taskset -cp 1 1234
+
+Ditto for the second real time task (with the remaining 25% of cache):
+
+# mkdir p1
+# echo "L3:0=7c00;1=fffff" > p1/schemata
+# echo 5678 > p1/tasks
+# taskset -cp 2 5678
-- 
2.5.0

[toc] | [prev] | [next] | [standalone]

#1501224 — [PATCH v4 17/18] x86/intel_rdt: Add scheduler hook

From: Fenghua Yu <fenghua.yu@intel.com>

Hook the x86 scheduler code to update closid based on whether the current
task is assigned to a specific closid or running on a CPU assigned to a
specific closid.

Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Signed-off-by: Tony Luck <tony.luck@intel.com>
---
 arch/x86/include/asm/intel_rdt.h         | 41 ++++++++++++++++++++++++++++++++
 arch/x86/kernel/cpu/intel_rdt.c          |  1 -
 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c |  3 +++
 arch/x86/kernel/process_32.c             |  4 ++++
 arch/x86/kernel/process_64.c             |  4 ++++
 5 files changed, 52 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/intel_rdt.h b/arch/x86/include/asm/intel_rdt.h
index 8435ec6..863e838 100644
--- a/arch/x86/include/asm/intel_rdt.h
+++ b/arch/x86/include/asm/intel_rdt.h
@@ -4,6 +4,9 @@
 #define IA32_L3_CBM_BASE	0xc90
 #define IA32_L2_CBM_BASE	0xd10
 
+#ifdef CONFIG_INTEL_RDT_A
+#include <asm/intel_rdt_common.h>
+
 /**
  * struct rdtgroup - store rdtgroup's data in resctrl file system.
  * @kn:				kernfs node
@@ -162,4 +165,42 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,
 				char *buf, size_t nbytes, loff_t off);
 int rdtgroup_schemata_show(struct kernfs_open_file *of,
 			   struct seq_file *s, void *v);
+
+/*
+ * intel_rdt_sched_in() - Writes the task's CLOSid to IA32_PQR_MSR
+ *
+ * Following considerations are made so that this has minimal impact
+ * on scheduler hot path:
+ * - This will stay as no-op unless we are running on an Intel SKU
+ *   which supports resource control and we enable by mounting the
+ *   resctrl file system.
+ * - Caches the per cpu CLOSid values and does the MSR write only
+ *   when a task with a different CLOSid is scheduled in.
+ */
+static inline void intel_rdt_sched_in(void)
+{
+	if (static_branch_likely(&rdt_enable_key)) {
+		struct intel_pqr_state *state = this_cpu_ptr(&pqr_state);
+		int closid;
+
+		/*
+		 * If this task has a closid assigned, use it.
+		 * Else use the closid assigned to this cpu.
+		 */
+		closid = current->closid;
+		if (closid == 0)
+			closid = this_cpu_read(cpu_closid);
+
+		if (closid != state->closid) {
+			state->closid = closid;
+			wrmsr(MSR_IA32_PQR_ASSOC, state->rmid, closid);
+		}
+	}
+}
+
+#else
+
+static inline void intel_rdt_sched_in(void) {}
+
+#endif /* CONFIG_INTEL_RDT_A */
 #endif /* _ASM_X86_INTEL_RDT_H */
diff --git a/arch/x86/kernel/cpu/intel_rdt.c b/arch/x86/kernel/cpu/intel_rdt.c
index 556a426..933bcb4 100644
--- a/arch/x86/kernel/cpu/intel_rdt.c
+++ b/arch/x86/kernel/cpu/intel_rdt.c
@@ -29,7 +29,6 @@
 #include <linux/cacheinfo.h>
 #include <linux/cpuhotplug.h>
 
-#include <asm/intel_rdt_common.h>
 #include <asm/intel-family.h>
 #include <asm/intel_rdt.h>
 
diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
index 6787fd8..62883af 100644
--- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
+++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
@@ -299,6 +299,9 @@ static void move_myself(struct callback_head *head)
 		kfree(rdtgrp);
 	}
 
+	/* update PQR_ASSOC MSR to make resource group go into effect */
+	intel_rdt_sched_in();
+
 	kfree(callback);
 }
 
diff --git a/arch/x86/kernel/process_32.c b/arch/x86/kernel/process_32.c
index d86be29..5495527 100644
--- a/arch/x86/kernel/process_32.c
+++ b/arch/x86/kernel/process_32.c
@@ -54,6 +54,7 @@
 #include <asm/debugreg.h>
 #include <asm/switch_to.h>
 #include <asm/vm86.h>
+#include <asm/intel_rdt.h>
 
 asmlinkage void ret_from_fork(void) __asm__("ret_from_fork");
 asmlinkage void ret_from_kernel_thread(void) __asm__("ret_from_kernel_thread");
@@ -314,5 +315,8 @@ __switch_to(struct task_struct *prev_p, struct task_struct *next_p)
 
 	this_cpu_write(current_task, next_p);
 
+	/* Load the Intel cache allocation PQR MSR. */
+	intel_rdt_sched_in();
+
 	return prev_p;
 }
diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c
index 63236d8..cdea7e3 100644
--- a/arch/x86/kernel/process_64.c
+++ b/arch/x86/kernel/process_64.c
@@ -49,6 +49,7 @@
 #include <asm/debugreg.h>
 #include <asm/switch_to.h>
 #include <asm/xen/hypervisor.h>
+#include <asm/intel_rdt.h>
 
 asmlinkage extern void ret_from_fork(void);
 
@@ -472,6 +473,9 @@ __switch_to(struct task_struct *prev_p, struct task_struct *next_p)
 			loadsegment(ss, __KERNEL_DS);
 	}
 
+	/* Load the Intel cache allocation PQR MSR. */
+	intel_rdt_sched_in();
+
 	return prev_p;
 }
 
-- 
2.5.0

[toc] | [prev] | [next] | [standalone]

#1501225 — [PATCH v4 14/18] x86/intel_rdt: Add cpus file

From: Tony Luck <tony.luck@intel.com>

Now we populate each directory with a read/write (mode 0644) file
named "cpus". This is used to over-ride the resources available
to processes in the default resource group when running on specific
CPUs.  Each "cpus" file reads as a cpumask showing which CPUs belong
to this resource group. Initially all online CPUs are assigned to
the default group. They can be added to other groups by writing a
cpumask to the "cpus" file in the directory for the resource group
(which will remove them from the previous group to which they were
assigned). CPU online/offline operations will delete CPUs that go
offline from whatever group they are in and add new CPUs to the
default group.

If there are CPUs assigned to a group when the directory is removed,
they are returned to the default group.

Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Signed-off-by: Tony Luck <tony.luck@intel.com>
---
 arch/x86/include/asm/intel_rdt.h         |   5 ++
 arch/x86/kernel/cpu/intel_rdt.c          |  12 +++
 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 127 +++++++++++++++++++++++++++++++
 3 files changed, 144 insertions(+)

diff --git a/arch/x86/include/asm/intel_rdt.h b/arch/x86/include/asm/intel_rdt.h
index 7eb8078..6ab31ba 100644
--- a/arch/x86/include/asm/intel_rdt.h
+++ b/arch/x86/include/asm/intel_rdt.h
@@ -9,13 +9,16 @@
  * @kn:				kernfs node
  * @rdtgroup_list:		linked list for all rdtgroups
  * @closid:			closid for this rdtgroup
+ * @cpu_mask:			CPUs assigned to this rdtgroup
  * @flags:			status bits
  * @waitcount:			how many cpus expect to find this
+ *				group when they acquire rdtgroup_mutex
  */
 struct rdtgroup {
 	struct kernfs_node	*kn;
 	struct list_head	rdtgroup_list;
 	int			closid;
+	struct cpumask		cpu_mask;
 	int			flags;
 	atomic_t		waitcount;
 };
@@ -148,6 +151,8 @@ union cpuid_0x10_1_edx {
 	unsigned int full;
 };
 
+DECLARE_PER_CPU_READ_MOSTLY(int, cpu_closid);
+
 void rdt_cbm_update(void *arg);
 struct rdtgroup *rdtgroup_kn_lock_live(struct kernfs_node *kn);
 void rdtgroup_kn_unlock(struct kernfs_node *kn);
diff --git a/arch/x86/kernel/cpu/intel_rdt.c b/arch/x86/kernel/cpu/intel_rdt.c
index 3d4a125..556a426 100644
--- a/arch/x86/kernel/cpu/intel_rdt.c
+++ b/arch/x86/kernel/cpu/intel_rdt.c
@@ -36,6 +36,8 @@
 /* Mutex to protect rdtgroup access. */
 DEFINE_MUTEX(rdtgroup_mutex);
 
+DEFINE_PER_CPU_READ_MOSTLY(int, cpu_closid);
+
 #define domain_init(name) LIST_HEAD_INIT(rdt_resources_all[name].domains)
 
 struct rdt_resource rdt_resources_all[] = {
@@ -242,8 +244,11 @@ static int intel_rdt_online_cpu(unsigned int cpu)
 	struct intel_pqr_state *state = this_cpu_ptr(&pqr_state);
 
 	mutex_lock(&rdtgroup_mutex);
+	per_cpu(cpu_closid, cpu) = 0;
 	for_each_rdt_resource(r)
 		update_domain(cpu, r, 1);
+	/* The cpu is set in default rdtgroup after online. */
+	cpumask_set_cpu(cpu, &rdtgroup_default.cpu_mask);
 	state->closid = 0;
 	wrmsr(MSR_IA32_PQR_ASSOC, state->rmid, 0);
 	mutex_unlock(&rdtgroup_mutex);
@@ -254,10 +259,17 @@ static int intel_rdt_online_cpu(unsigned int cpu)
 static int intel_rdt_offline_cpu(unsigned int cpu)
 {
 	struct rdt_resource *r;
+	struct rdtgroup *rdtgrp;
+	struct list_head *l;
 
 	mutex_lock(&rdtgroup_mutex);
 	for_each_rdt_resource(r)
 		update_domain(cpu, r, 0);
+	list_for_each(l, &rdt_all_groups) {
+		rdtgrp = list_entry(l, struct rdtgroup, rdtgroup_list);
+		if (cpumask_test_and_clear_cpu(cpu, &rdtgrp->cpu_mask))
+			break;
+	}
 	mutex_unlock(&rdtgroup_mutex);
 
 	return 0;
diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
index 9e0044d..f2d7a3a 100644
--- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
+++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
@@ -20,6 +20,7 @@
 
 #define pr_fmt(fmt)	KBUILD_MODNAME ": " fmt
 
+#include <linux/cpu.h>
 #include <linux/fs.h>
 #include <linux/sysfs.h>
 #include <linux/kernfs.h>
@@ -180,6 +181,117 @@ static struct kernfs_ops rdtgroup_kf_single_ops = {
 	.seq_show		= rdtgroup_seqfile_show,
 };
 
+static int rdtgroup_cpus_show(struct kernfs_open_file *of,
+			      struct seq_file *s, void *v)
+{
+	struct rdtgroup *rdtgrp;
+	int ret = 0;
+
+	rdtgrp = rdtgroup_kn_lock_live(of->kn);
+
+	if (rdtgrp)
+		seq_printf(s, "%*pb\n", cpumask_pr_args(&rdtgrp->cpu_mask));
+	else
+		ret = -ENOENT;
+	rdtgroup_kn_unlock(of->kn);
+
+	return ret;
+}
+
+static ssize_t rdtgroup_cpus_write(struct kernfs_open_file *of,
+				   char *buf, size_t nbytes, loff_t off)
+{
+	struct rdtgroup *rdtgrp, *r;
+	cpumask_var_t tmpmask, newmask;
+	int ret, cpu;
+
+	if (!buf)
+		return -EINVAL;
+
+	if (!alloc_cpumask_var(&tmpmask, GFP_KERNEL))
+		return -ENOMEM;
+	if (!alloc_cpumask_var(&newmask, GFP_KERNEL)) {
+		free_cpumask_var(tmpmask);
+		return -ENOMEM;
+	}
+	rdtgrp = rdtgroup_kn_lock_live(of->kn);
+	if (!rdtgrp) {
+		ret = -ENOENT;
+		goto unlock;
+	}
+
+	ret = cpumask_parse(buf, newmask);
+	if (ret)
+		goto unlock;
+
+	get_online_cpus();
+	/* check that user didn't specify any offline cpus */
+	cpumask_andnot(tmpmask, newmask, cpu_online_mask);
+	if (cpumask_weight(tmpmask)) {
+		ret = -EINVAL;
+		goto end;
+	}
+
+	/* Are trying to drop some cpus from this group? */
+	cpumask_andnot(tmpmask, &rdtgrp->cpu_mask, newmask);
+	if (cpumask_weight(tmpmask)) {
+		/* Can't drop from default group */
+		if (rdtgrp == &rdtgroup_default) {
+			ret = -EINVAL;
+			goto end;
+		}
+		/* Give any dropped cpus to rdtgroup_default */
+		cpumask_or(&rdtgroup_default.cpu_mask,
+			   &rdtgroup_default.cpu_mask, tmpmask);
+		for_each_cpu(cpu, tmpmask)
+			per_cpu(cpu_closid, cpu) = 0;
+	}
+
+	/*
+	 * If we added cpus, remove them from previous group that owned them
+	 * and update per-cpu rdtgroup pointers to refer to us
+	 */
+	cpumask_andnot(tmpmask, newmask, &rdtgrp->cpu_mask);
+	if (cpumask_weight(tmpmask)) {
+		struct list_head *l;
+
+		list_for_each(l, &rdt_all_groups) {
+			r = list_entry(l, struct rdtgroup, rdtgroup_list);
+			if (r == rdtgrp)
+				continue;
+			cpumask_andnot(&r->cpu_mask, &r->cpu_mask, tmpmask);
+		}
+		for_each_cpu(cpu, tmpmask)
+			per_cpu(cpu_closid, cpu) = rdtgrp->closid;
+	}
+
+	/* Done pushing/pulling - update this group with new mask */
+	cpumask_copy(&rdtgrp->cpu_mask, newmask);
+
+end:
+	put_online_cpus();
+unlock:
+	rdtgroup_kn_unlock(of->kn);
+	free_cpumask_var(tmpmask);
+	free_cpumask_var(newmask);
+
+	return ret ?: nbytes;
+}
+
+/* Files in each rdtgroup */
+static struct rftype rdtgroup_base_files[] = {
+	{
+		.name		= "cpus",
+		.mode		= 0644,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.write		= rdtgroup_cpus_write,
+		.seq_show	= rdtgroup_cpus_show,
+	},
+	{
+		/* NULL terminated */
+	}
+};
+
 static int rdt_num_closid_show(struct kernfs_open_file *of,
 			       struct seq_file *seq, void *v)
 {
@@ -545,6 +657,10 @@ static int rdtgroup_mkdir(struct kernfs_node *parent_kn, const char *name,
 	if (ret)
 		goto out_destroy;
 
+	ret = rdtgroup_add_files(kn, rdtgroup_base_files);
+	if (ret)
+		goto out_destroy;
+
 	kernfs_activate(kn);
 
 	ret = 0;
@@ -582,6 +698,10 @@ static int rdtgroup_rmdir(struct kernfs_node *kn)
 		return -EPERM;
 	}
 
+	/* Give any CPUs back to the default group */
+	cpumask_or(&rdtgroup_default.cpu_mask,
+		   &rdtgroup_default.cpu_mask, &rdtgrp->cpu_mask);
+
 	rdtgrp->flags = RDT_DELETED;
 	closid_free(rdtgrp->closid);
 	list_del(&rdtgrp->rdtgroup_list);
@@ -618,11 +738,18 @@ static int __init rdtgroup_setup_root(void)
 	rdtgroup_default.closid = 0;
 	list_add(&rdtgroup_default.rdtgroup_list, &rdt_all_groups);
 
+	ret = rdtgroup_add_files(rdt_root->kn, rdtgroup_base_files);
+	if (ret) {
+		kernfs_destroy_root(rdt_root);
+		goto out;
+	}
+
 	rdtgroup_default.kn = rdt_root->kn;
 	ret = rdtgroup_create_info_dir(rdtgroup_default.kn);
 	if (!ret)
 		kernfs_activate(rdtgroup_default.kn);
 
+out:
 	mutex_unlock(&rdtgroup_mutex);
 
 	return 0;
-- 
2.5.0

[toc] | [prev] | [next] | [standalone]

#1502489 — Re: [PATCH v4 14/18] x86/intel_rdt: Add cpus file

On Fri, 14 Oct 2016, Fenghua Yu wrote:
>  static int intel_rdt_offline_cpu(unsigned int cpu)
>  {
>  	struct rdt_resource *r;
> +	struct rdtgroup *rdtgrp;
> +	struct list_head *l;
>  
>  	mutex_lock(&rdtgroup_mutex);
>  	for_each_rdt_resource(r)
>  		update_domain(cpu, r, 0);
> +	list_for_each(l, &rdt_all_groups) {

list_for_each_entry ...

> +		rdtgrp = list_entry(l, struct rdtgroup, rdtgroup_list);
> +		if (cpumask_test_and_clear_cpu(cpu, &rdtgrp->cpu_mask))
> +			break;

> +static ssize_t rdtgroup_cpus_write(struct kernfs_open_file *of,
> +				   char *buf, size_t nbytes, loff_t off)
> +{
     ....
> +	/* Are trying to drop some cpus from this group? */

  	/* Check whether cpus are dropped from this group */

> +	cpumask_andnot(tmpmask, &rdtgrp->cpu_mask, newmask);
> +	if (cpumask_weight(tmpmask)) {
> +		/* Can't drop from default group */
> +		if (rdtgrp == &rdtgroup_default) {
> +			ret = -EINVAL;
> +			goto end;
> +		}
> +		/* Give any dropped cpus to rdtgroup_default */
> +		cpumask_or(&rdtgroup_default.cpu_mask,
> +			   &rdtgroup_default.cpu_mask, tmpmask);
> +		for_each_cpu(cpu, tmpmask)
> +			per_cpu(cpu_closid, cpu) = 0;
> +	}
> +
> +	/*
> +	 * If we added cpus, remove them from previous group that owned them
> +	 * and update per-cpu rdtgroup pointers to refer to us

s/per-cpu rdtgroup pointers to refer to us/the per-cpu closid/

> +	 */
> +	cpumask_andnot(tmpmask, newmask, &rdtgrp->cpu_mask);
> +	if (cpumask_weight(tmpmask)) {
> +		struct list_head *l;
> +
> +		list_for_each(l, &rdt_all_groups) {
> +			r = list_entry(l, struct rdtgroup, rdtgroup_list);

Once more: list_for_each_entry()

> @@ -582,6 +698,10 @@ static int rdtgroup_rmdir(struct kernfs_node *kn)
>  		return -EPERM;
>  	}
>  
> +	/* Give any CPUs back to the default group */
> +	cpumask_or(&rdtgroup_default.cpu_mask,
> +		   &rdtgroup_default.cpu_mask, &rdtgrp->cpu_mask);

What resets the per-cpu closid to 0?

Thanks,

	tglx

[toc] | [prev] | [next] | [standalone]

#1501226 — [PATCH v4 12/18] x86/intel_rdt: Add "info" files to resctrl file system

From: Fenghua Yu <fenghua.yu@intel.com>

For the convenience of applications we make the decoded values of some
of the CPUID values available in read-only (0444) files.

Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Signed-off-by: Tony Luck <tony.luck@intel.com>
---
 arch/x86/include/asm/intel_rdt.h         |  24 +++++
 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 175 ++++++++++++++++++++++++++++++-
 2 files changed, 198 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/intel_rdt.h b/arch/x86/include/asm/intel_rdt.h
index f7acae2..ea8c09b3 100644
--- a/arch/x86/include/asm/intel_rdt.h
+++ b/arch/x86/include/asm/intel_rdt.h
@@ -22,6 +22,30 @@ extern struct list_head rdt_all_groups;
 int __init rdtgroup_init(void);
 
 /**
+ * struct rftype - describe each file in the resctrl file system
+ * @name: file name
+ * @mode: access mode
+ * @kf_ops: operations
+ * @seq_show: show content of the file
+ * @write: write to the file
+ */
+struct rftype {
+	char			*name;
+	umode_t			mode;
+	struct kernfs_ops	*kf_ops;
+
+	int (*seq_show)(struct kernfs_open_file *of,
+			struct seq_file *sf, void *v);
+	/*
+	 * write() is the generic write callback which maps directly to
+	 * kernfs write operation and overrides all other operations.
+	 * Maximum write size is determined by ->max_write_len.
+	 */
+	ssize_t (*write)(struct kernfs_open_file *of,
+			 char *buf, size_t nbytes, loff_t off);
+};
+
+/**
  * struct rdt_resource - attributes of an RDT resource
  * @enabled:			Is this feature enabled on this machine
  * @name:			Name to use in "schemata" file
diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
index b2bbd04..316fa0c 100644
--- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
+++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
@@ -23,6 +23,8 @@
 #include <linux/fs.h>
 #include <linux/sysfs.h>
 #include <linux/kernfs.h>
+#include <linux/seq_file.h>
+#include <linux/sched.h>
 #include <linux/slab.h>
 
 #include <uapi/linux/magic.h>
@@ -34,6 +36,173 @@ struct kernfs_root *rdt_root;
 struct rdtgroup rdtgroup_default;
 LIST_HEAD(rdt_all_groups);
 
+/* set uid and gid of rdtgroup dirs and files to that of the creator */
+static int rdtgroup_kn_set_ugid(struct kernfs_node *kn)
+{
+	struct iattr iattr = { .ia_valid = ATTR_UID | ATTR_GID,
+				.ia_uid = current_fsuid(),
+				.ia_gid = current_fsgid(), };
+
+	if (uid_eq(iattr.ia_uid, GLOBAL_ROOT_UID) &&
+	    gid_eq(iattr.ia_gid, GLOBAL_ROOT_GID))
+		return 0;
+
+	return kernfs_setattr(kn, &iattr);
+}
+
+static int rdtgroup_add_file(struct kernfs_node *parent_kn, struct rftype *rft)
+{
+	struct kernfs_node *kn;
+	int ret;
+
+	kn = __kernfs_create_file(parent_kn, rft->name, rft->mode,
+				  0, rft->kf_ops, rft, NULL, NULL);
+	if (IS_ERR(kn))
+		return PTR_ERR(kn);
+
+	ret = rdtgroup_kn_set_ugid(kn);
+	if (ret) {
+		kernfs_remove(kn);
+		return ret;
+	}
+
+	return 0;
+}
+
+static int rdtgroup_add_files(struct kernfs_node *kn, struct rftype *rfts)
+{
+	struct rftype *rft;
+	int ret;
+
+	lockdep_assert_held(&rdtgroup_mutex);
+
+	for (rft = rfts; rft->name; rft++) {
+		ret = rdtgroup_add_file(kn, rft);
+		if (ret)
+			goto error;
+	}
+
+	return 0;
+error:
+	pr_warn("%s: failed to add %s, err=%d\n", __func__, rft->name, ret);
+	while (--rft >= rfts)
+		kernfs_remove_by_name(kn, rft->name);
+	return ret;
+}
+
+static int rdtgroup_seqfile_show(struct seq_file *m, void *arg)
+{
+	struct kernfs_open_file *of = m->private;
+	struct rftype *rft = of->kn->priv;
+
+	if (rft->seq_show)
+		return rft->seq_show(of, m, arg);
+	return 0;
+}
+
+static ssize_t rdtgroup_file_write(struct kernfs_open_file *of, char *buf,
+				   size_t nbytes, loff_t off)
+{
+	struct rftype *rft = of->kn->priv;
+
+	if (rft->write)
+		return rft->write(of, buf, nbytes, off);
+
+	return -EINVAL;
+}
+
+static struct kernfs_ops rdtgroup_kf_single_ops = {
+	.atomic_write_len	= PAGE_SIZE,
+	.write			= rdtgroup_file_write,
+	.seq_show		= rdtgroup_seqfile_show,
+};
+
+static int rdt_num_closid_show(struct kernfs_open_file *of,
+			       struct seq_file *seq, void *v)
+{
+	struct rdt_resource *r = of->kn->parent->priv;
+
+	seq_printf(seq, "%d\n", r->num_closid);
+
+	return 0;
+}
+
+static int rdt_cbm_val_show(struct kernfs_open_file *of,
+			    struct seq_file *seq, void *v)
+{
+	struct rdt_resource *r = of->kn->parent->priv;
+
+	seq_printf(seq, "%x\n", r->max_cbm);
+
+	return 0;
+}
+
+/* rdtgroup information files for one cache resource. */
+static struct rftype res_info_files[] = {
+	{
+		.name		= "num_closid",
+		.mode		= 0444,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdt_num_closid_show,
+	},
+	{
+		.name		= "cbm_val",
+		.mode		= 0444,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdt_cbm_val_show,
+	},
+	{
+		/* NULL terminated */
+	}
+};
+
+static int rdtgroup_create_info_dir(struct kernfs_node *parent_kn)
+{
+	struct kernfs_node *kn, *kn_subdir;
+	struct rdt_resource *r;
+	int ret;
+
+	/* create the directory */
+	kn = kernfs_create_dir(parent_kn, "info", parent_kn->mode, NULL);
+	if (IS_ERR(kn))
+		return PTR_ERR(kn);
+	kernfs_get(kn);
+
+	for_each_rdt_resource(r) {
+		kn_subdir = kernfs_create_dir(kn, r->name, kn->mode, r);
+		if (IS_ERR(kn_subdir)) {
+			ret = PTR_ERR(kn_subdir);
+			goto out_destroy;
+		}
+		kernfs_get(kn_subdir);
+		ret = rdtgroup_kn_set_ugid(kn_subdir);
+		if (ret)
+			goto out_destroy;
+		ret = rdtgroup_add_files(kn_subdir, res_info_files);
+		if (ret)
+			goto out_destroy;
+		kernfs_activate(kn_subdir);
+	}
+
+	/*
+	 * This extra ref will be put in kernfs_remove() and guarantees
+	 * that @rdtgrp->kn is always accessible.
+	 */
+	kernfs_get(kn);
+
+	ret = rdtgroup_kn_set_ugid(kn);
+	if (ret)
+		goto out_destroy;
+
+	kernfs_activate(kn);
+
+	return 0;
+
+out_destroy:
+	kernfs_remove(kn);
+	return ret;
+}
+
 static void l3_qos_cfg_update(void *arg)
 {
 	struct rdt_resource *r = arg;
@@ -181,6 +350,8 @@ static struct kernfs_syscall_ops rdtgroup_kf_syscall_ops = {
 
 static int __init rdtgroup_setup_root(void)
 {
+	int ret;
+
 	rdt_root = kernfs_create_root(&rdtgroup_kf_syscall_ops,
 				      KERNFS_ROOT_CREATE_DEACTIVATED,
 				      &rdtgroup_default);
@@ -193,7 +364,9 @@ static int __init rdtgroup_setup_root(void)
 	list_add(&rdtgroup_default.rdtgroup_list, &rdt_all_groups);
 
 	rdtgroup_default.kn = rdt_root->kn;
-	kernfs_activate(rdtgroup_default.kn);
+	ret = rdtgroup_create_info_dir(rdtgroup_default.kn);
+	if (!ret)
+		kernfs_activate(rdtgroup_default.kn);
 
 	mutex_unlock(&rdtgroup_mutex);
 
-- 
2.5.0

[toc] | [prev] | [next] | [standalone]

#1502387 — Re: [PATCH v4 12/18] x86/intel_rdt: Add "info" files to resctrl file system

On Fri, 14 Oct 2016, Fenghua Yu wrote:
>  static int __init rdtgroup_setup_root(void)
>  {
> +	int ret;
> +
>  	rdt_root = kernfs_create_root(&rdtgroup_kf_syscall_ops,
>  				      KERNFS_ROOT_CREATE_DEACTIVATED,
>  				      &rdtgroup_default);
> @@ -193,7 +364,9 @@ static int __init rdtgroup_setup_root(void)
>  	list_add(&rdtgroup_default.rdtgroup_list, &rdt_all_groups);
>  
>  	rdtgroup_default.kn = rdt_root->kn;
> -	kernfs_activate(rdtgroup_default.kn);
> +	ret = rdtgroup_create_info_dir(rdtgroup_default.kn);
> +	if (!ret)
> +		kernfs_activate(rdtgroup_default.kn);
>  
>  	mutex_unlock(&rdtgroup_mutex);

So this is followed by:

   	return 0;

which means that an error in rdtgroup_create_info_dir() is ignored. As a
consequence the mount point is created and the file system is registered
w/o the info directory....

Thanks,

	tglx

[toc] | [prev] | [next] | [standalone]

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

Back to top | Article view | linux.kernel

csiph-web