Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1502568

Re: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file system

From Thomas Gleixner <tglx@linutronix.de>
Newsgroups linux.kernel
Subject Re: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file system
Date 2016-10-18 01:00 +0200
Message-ID <stp2V-1JC-1@gated-at.bofh.it> (permalink)
References <ssjVD-7Za-5@gated-at.bofh.it> <ssjVD-7Za-15@gated-at.bofh.it> <stnua-De-17@gated-at.bofh.it> <sto6R-YG-3@gated-at.bofh.it>
Organization linux.* mail to news gateway

Show all headers | View raw

On Mon, 17 Oct 2016, Luck, Tony wrote:
> On Mon, Oct 17, 2016 at 11:14:55PM +0200, Thomas Gleixner wrote:
> > > +	/* Compute rdt_max_closid across all resources */
> > > +	rdt_max_closid = 0;
> > > +	for_each_rdt_resource(r)
> > > +		rdt_max_closid = max(rdt_max_closid, r->num_closid);
> > 
> > Oh no! This needs to be min().
> > 
> > Assume you have a system with L3 and L2 CAT. L2 reports COS_MAX=16, L3
> > reports COS_MAX=16 as well. Then you enabled CDP which cuts L3 COS_MAX in
> > half. So the real usable number of CLOSIDs is going to be 8 for both L2 and
> > L3 simply because you do not have a seperation of L2 and L3 in
> > MSR_PQR_ASSOC. And if you allow 16 then any CLOSID > 8 will result in
> > undefined behaviour. See SDM:
> > 
> >  "When CDP is enabled, specifying a COS value in IA32_PQR_ASSOC.COS outside
> >   of the lower half of the COS space will cause undefined performance impact
> >   to code and data fetches due to MSR space re-indexing into code/data masks
> >   when CDP is enabled."
> 
> Bother.  The SDM also has this gem:
> 
> 17.17.5.1 Cache Allocation Technology Dynamic Configuration
> Both the CAT masks and CQM registers are accessible and modifiable at
> any time during execution using RDMSR/WRMSR unless otherwise noted. When
> writing to these MSRs a #GP(0) will be generated if any of the following
> conditions occur:
> 
>  * Writing a COS greater than the supported maximum (specified as the
>    maximum value of CPUID.(EAX=10H, ECX=ResID):EDX[15:0] for all valid
>    ResID values) is written to the IA32_PQR_ASSOC.CLOS field.
> 
> With the intent here being that if you have more of one resource than
> another, you can use all of the resources in the larger (with the
> resource with fewer mask registers defaulting to the maximum value
> when PQR_ASSOC.COS is too large [and I can't find the text that talks
> about that default behaviour :-( ]
>
> I think this all means that L3/CDP is "special". If CDP is on, we can't
> use the top half of the CLOSID space that CPUID.(EAX=10H, ECX=1):EDX[15:0]
> told us is present. So we can't exceed the half-way point.

That's my understanding.

> But in other cases like L3 with max=16 and L2 with max=8 we should allow
> 16 groups, but 0-7 allow control of L3 and L2, while 8-15 only allow L3
> control (the schemata code enforces this when you get to that part).

Cute. Welcome to configuration hell!

So how are we going to deal with that in the schematas? Assume the L3=16
and L2=8 case(no CDP). So effectively any write of L2 to CLOSID=0 will
affect the setting of L2 in CLOSID=8.

Will the code tell the user that L2 cannot be set for CLOSID >= 8? 

Will it print the setting of CLOSID - 8 for CLOSID >= 8 when you read the
schemata file of CLOSID >= 8?

And of course this becomes very interesting when CLOSID 1 is deleted, then
what happens to CLOSID 9? Not to talk about the case when CLOSID 1 is
reused again.

Seems to be a well thought out hardware feature once again.

> Perhaps we can encode this in another field in the rdt_resource structure
> that says that some maximums globally override all others, while some
> can be legitimately exceeded.

That should work.
 
> I'll have some words with the h/w architect, and get the SDM fixed in
> the next edition.

Whatever the outcome will be :)

Thanks,

	tglx

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH v4 00/18] Intel Cache Allocation Technology "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
  [PATCH v4 02/18] cacheinfo: Introduce cache id "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
    Re: [PATCH v4 02/18] cacheinfo: Introduce cache id Thomas Gleixner <tglx@linutronix.de> - 2016-10-17 12:40 +0200
  [PATCH v4 16/18] x86/intel_rdt: Add schemata file "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
    Re: [PATCH v4 16/18] x86/intel_rdt: Add schemata file Thomas Gleixner <tglx@linutronix.de> - 2016-10-18 00:40 +0200
  [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file system "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
    Re: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file  system Thomas Gleixner <tglx@linutronix.de> - 2016-10-17 23:20 +0200
      Re: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file system "Luck, Tony" <tony.luck@intel.com> - 2016-10-18 00:00 +0200
        Re: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file  system Thomas Gleixner <tglx@linutronix.de> - 2016-10-18 01:00 +0200
          RE: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file  system Thomas Gleixner <tglx@linutronix.de> - 2016-10-18 01:10 +0200
            RE: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file system "Luck, Tony" <tony.luck@intel.com> - 2016-10-18 01:20 +0200
              RE: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file  system Thomas Gleixner <tglx@linutronix.de> - 2016-10-18 01:30 +0200
          RE: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file system "Luck, Tony" <tony.luck@intel.com> - 2016-10-18 01:10 +0200
      Re: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file system Fenghua Yu <fenghua.yu@intel.com> - 2016-10-18 00:20 +0200
        Re: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file  system Thomas Gleixner <tglx@linutronix.de> - 2016-10-18 01:30 +0200
          Re: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file system "Luck, Tony" <tony.luck@intel.com> - 2016-10-18 01:40 +0200
            Re: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file system Fenghua Yu <fenghua.yu@intel.com> - 2016-10-18 02:00 +0200
            Re: [PATCH v4 13/18] x86/intel_rdt: Add mkdir to resctrl file  system Thomas Gleixner <tglx@linutronix.de> - 2016-10-18 12:50 +0200
  [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters from CPUID "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
    Re: [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters  from CPUID Thomas Gleixner <tglx@linutronix.de> - 2016-10-17 15:50 +0200
      Re: [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters  from CPUID Fenghua Yu <fenghua.yu@intel.com> - 2016-10-17 17:10 +0200
        Re: [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters  from CPUID "Luck, Tony" <tony.luck@intel.com> - 2016-10-17 18:40 +0200
          RE: [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters  from CPUID "Yu, Fenghua" <fenghua.yu@intel.com> - 2016-10-17 18:50 +0200
            Re: [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters  from CPUID "Luck, Tony" <tony.luck@intel.com> - 2016-10-17 22:30 +0200
          Re: [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters  from CPUID Thomas Gleixner <tglx@linutronix.de> - 2016-10-17 19:10 +0200
        Re: [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters  from CPUID Thomas Gleixner <tglx@linutronix.de> - 2016-10-17 19:10 +0200
        Re: [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters  from CPUID Thomas Gleixner <tglx@linutronix.de> - 2016-10-17 19:10 +0200
          Re: [PATCH v4 08/18] x86/intel_rdt: Pick up L3/L2 RDT parameters  from CPUID Fenghua Yu <fenghua.yu@intel.com> - 2016-10-17 20:20 +0200
  [PATCH v4 01/18] Documentation, ABI: Add a document entry for cache id "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
    Re: [PATCH v4 01/18] Documentation, ABI: Add a document entry for  cache id Thomas Gleixner <tglx@linutronix.de> - 2016-10-17 12:40 +0200
  [PATCH v4 18/18] MAINTAINERS: Add maintainer for Intel RDT resource allocation "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
  [PATCH v4 04/18] x86/intel_rdt: Feature discovery "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
  [PATCH v4 11/18] x86/intel_rdt: Add basic resctrl filesystem support "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
    Re: [PATCH v4 11/18] x86/intel_rdt: Add basic resctrl filesystem  support Thomas Gleixner <tglx@linutronix.de> - 2016-10-17 21:40 +0200
  [PATCH v4 05/18] Documentation, x86: Documentation for Intel resource allocation user interface "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
  [PATCH v4 17/18] x86/intel_rdt: Add scheduler hook "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
  [PATCH v4 14/18] x86/intel_rdt: Add cpus file "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
    Re: [PATCH v4 14/18] x86/intel_rdt: Add cpus file Thomas Gleixner <tglx@linutronix.de> - 2016-10-17 23:40 +0200
  [PATCH v4 12/18] x86/intel_rdt: Add "info" files to resctrl file system "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
    Re: [PATCH v4 12/18] x86/intel_rdt: Add "info" files to resctrl file  system Thomas Gleixner <tglx@linutronix.de> - 2016-10-17 21:50 +0200
  [PATCH v4 15/18] x86/intel_rdt: Add tasks files "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
    Re: [PATCH v4 15/18] x86/intel_rdt: Add tasks files Thomas Gleixner <tglx@linutronix.de> - 2016-10-18 00:10 +0200
      Re: [PATCH v4 15/18] x86/intel_rdt: Add tasks files "Luck, Tony" <tony.luck@intel.com> - 2016-10-18 00:20 +0200
  [PATCH v4 10/18] x86/intel_rdt: Build structures for each resource based on cache topology "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
    Re: [PATCH v4 10/18] x86/intel_rdt: Build structures for each resource  based on cache topology Thomas Gleixner <tglx@linutronix.de> - 2016-10-17 16:50 +0200
  [PATCH v4 03/18] x86, intel_cacheinfo: Enable cache id in x86 "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
    Re: [PATCH v4 03/18] x86, intel_cacheinfo: Enable cache id in x86 Thomas Gleixner <tglx@linutronix.de> - 2016-10-17 13:00 +0200
  [PATCH v4 07/18] x86/intel_rdt: Add Haswell feature discovery "Fenghua Yu" <fenghua.yu@intel.com> - 2016-10-15 01:20 +0200
    Re: [PATCH v4 07/18] x86/intel_rdt: Add Haswell feature discovery Thomas Gleixner <tglx@linutronix.de> - 2016-10-17 13:10 +0200

csiph-web