Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1200246 > unrolled thread

[PATCH v2 1/3] cgroup: define controller file conventions

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  [PATCH v2 1/3] cgroup: define controller file conventions Tejun Heo <tj@kernel.org> - 2015-08-04 21:40 +0200
    Re: [PATCH v2 1/3] cgroup: define controller file conventions Kamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> - 2015-08-05 03:00 +0200
      Re: [PATCH v2 1/3] cgroup: define controller file conventions Michal Hocko <mhocko@kernel.org> - 2015-08-05 09:50 +0200
        Re: [PATCH v2 1/3] cgroup: define controller file conventions Kamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> - 2015-08-06 04:40 +0200
          Re: [PATCH v2 1/3] cgroup: define controller file conventions Michal Hocko <mhocko@kernel.org> - 2015-08-07 20:20 +0200
            Re: [PATCH v2 1/3] cgroup: define controller file conventions Johannes Weiner <hannes@cmpxchg.org> - 2015-08-18 00:10 +0200
    Re: [PATCH v2 1/3] cgroup: define controller file conventions Johannes Weiner <hannes@cmpxchg.org> - 2015-08-17 23:40 +0200

#1200246 — [PATCH v2 1/3] cgroup: define controller file conventions

From 6abc8ca19df0078de17dc38340db3002ed489ce7 Mon Sep 17 00:00:00 2001
From: Tejun Heo <tj@kernel.org>
Date: Tue, 4 Aug 2015 15:20:55 -0400

Traditionally, each cgroup controller implemented whatever interface
it wanted leading to interfaces which are widely inconsistent.
Examining the requirements of the controllers readily yield that there
are only a few control schemes shared among all.

Two major controllers already had to implement new interface for the
unified hierarchy due to significant structural changes.  Let's take
the chance to establish common conventions throughout all controllers.

This patch defines CGROUP_WEIGHT_MIN/DFL/MAX to be used on all weight
based control knobs and documents the conventions that controllers
should follow on the unified hierarchy.  Except for io.weight knob,
all existing unified hierarchy knobs are already compliant.  A
follow-up patch will update io.weight.

v2: Added descriptions of min, low and high knobs.

Signed-off-by: Tejun Heo <tj@kernel.org>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Cc: Li Zefan <lizefan@huawei.com>
Cc: Peter Zijlstra <peterz@infradead.org>
---
Hello,

Added low/high descriptions and applied to the following git branch.

 git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git for-4.3-unified-base

The branch currently only contains this patch and will stay stable so
that it can be pulled from.  I kept the base weight as DFL for now.
If we decide to change it, I'll apply the change on top.

Thanks.

 Documentation/cgroups/unified-hierarchy.txt | 80 ++++++++++++++++++++++++++---
 include/linux/cgroup.h                      |  9 ++++
 2 files changed, 81 insertions(+), 8 deletions(-)

diff --git a/Documentation/cgroups/unified-hierarchy.txt b/Documentation/cgroups/unified-hierarchy.txt
index 86847a7..1ee9caf 100644
--- a/Documentation/cgroups/unified-hierarchy.txt
+++ b/Documentation/cgroups/unified-hierarchy.txt
@@ -23,10 +23,13 @@ CONTENTS
 5. Other Changes
   5-1. [Un]populated Notification
   5-2. Other Core Changes
-  5-3. Per-Controller Changes
-    5-3-1. blkio
-    5-3-2. cpuset
-    5-3-3. memory
+  5-3. Controller File Conventions
+    5-3-1. Format
+    5-3-2. Control Knobs
+  5-4. Per-Controller Changes
+    5-4-1. blkio
+    5-4-2. cpuset
+    5-4-3. memory
 6. Planned Changes
   6-1. CAP for resource control
 
@@ -372,14 +375,75 @@ supported and the interface files "release_agent" and
 - The "cgroup.clone_children" file is removed.
 
 
-5-3. Per-Controller Changes
+5-3. Controller File Conventions
 
-5-3-1. blkio
+5-3-1. Format
+
+In general, all controller files should be in one of the following
+formats whenever possible.
+
+- Values only files
+
+  VAL0 VAL1...\n
+
+- Flat keyed files
+
+  KEY0 VAL0\n
+  KEY1 VAL1\n
+  ...
+
+- Nested keyed files
+
+  KEY0 SUB_KEY0=VAL00 SUB_KEY1=VAL01...
+  KEY1 SUB_KEY0=VAL10 SUB_KEY1=VAL11...
+  ...
+
+For a writeable file, the format for writing should generally match
+reading; however, controllers may allow omitting later fields or
+implement restricted shortcuts for most common use cases.
+
+For both flat and nested keyed files, only the values for a single key
+can be written at a time.  For nested keyed files, the sub key pairs
+may be specified in any order and not all pairs have to be specified.
+
+
+5-3-2. Control Knobs
+
+- Settings for a single feature should generally be implemented in a
+  single file.
+
+- In general, the root cgroup should be exempt from resource control
+  and thus shouldn't have resource control knobs.
+
+- If a controller implements ratio based resource distribution, the
+  control knob should be named "weight" and have the range [1, 10000]
+  and 100 should be the default value.  The values are chosen to allow
+  enough and symmetric bias in both directions while keeping it
+  intuitive (the default is 100%).
+
+- If a controller implements an absolute resource guarantee and/or
+  limit, the control knobs should be named "min" and "max"
+  respectively.  If a controller implements best effort resource
+  gurantee and/or limit, the control knobs should be named "low" and
+  "high" respectively.
+
+  In the above four control files, the special token "max" should be
+  used to represent upward infinity for both reading and writing.
+
+- If a setting has configurable default value and specific overrides,
+  the default settings should be keyed with "default" and appear as
+  the first entry in the file.  Specific entries can use "default" as
+  its value to indicate inheritance of the default value.
+
+
+5-4. Per-Controller Changes
+
+5-4-1. blkio
 
 - blk-throttle becomes properly hierarchical.
 
 
-5-3-2. cpuset
+5-4-2. cpuset
 
 - Tasks are kept in empty cpusets after hotplug and take on the masks
   of the nearest non-empty ancestor, instead of being moved to it.
@@ -388,7 +452,7 @@ supported and the interface files "release_agent" and
   masks of the nearest non-empty ancestor.
 
 
-5-3-3. memory
+5-4-3. memory
 
 - use_hierarchy is on by default and the cgroup file for the flag is
   not created.
diff --git a/include/linux/cgroup.h b/include/linux/cgroup.h
index a593e29..c6bf9d3 100644
--- a/include/linux/cgroup.h
+++ b/include/linux/cgroup.h
@@ -22,6 +22,15 @@
 
 #ifdef CONFIG_CGROUPS
 
+/*
+ * All weight knobs on the default hierarhcy should use the following min,
+ * default and max values.  The default value is the logarithmic center of
+ * MIN and MAX and allows 100x to be expressed in both directions.
+ */
+#define CGROUP_WEIGHT_MIN		1
+#define CGROUP_WEIGHT_DFL		100
+#define CGROUP_WEIGHT_MAX		10000
+
 /* a css_task_iter should be treated as an opaque object */
 struct css_task_iter {
 	struct cgroup_subsys		*ss;
-- 
2.4.3

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1200333

On 2015/08/05 4:31, Tejun Heo wrote:
>  From 6abc8ca19df0078de17dc38340db3002ed489ce7 Mon Sep 17 00:00:00 2001
> From: Tejun Heo <tj@kernel.org>
> Date: Tue, 4 Aug 2015 15:20:55 -0400
>
> Traditionally, each cgroup controller implemented whatever interface
> it wanted leading to interfaces which are widely inconsistent.
> Examining the requirements of the controllers readily yield that there
> are only a few control schemes shared among all.
>
> Two major controllers already had to implement new interface for the
> unified hierarchy due to significant structural changes.  Let's take
> the chance to establish common conventions throughout all controllers.
>
> This patch defines CGROUP_WEIGHT_MIN/DFL/MAX to be used on all weight
> based control knobs and documents the conventions that controllers
> should follow on the unified hierarchy.  Except for io.weight knob,
> all existing unified hierarchy knobs are already compliant.  A
> follow-up patch will update io.weight.
>
> v2: Added descriptions of min, low and high knobs.
>
> Signed-off-by: Tejun Heo <tj@kernel.org>
> Acked-by: Johannes Weiner <hannes@cmpxchg.org>
> Cc: Li Zefan <lizefan@huawei.com>
> Cc: Peter Zijlstra <peterz@infradead.org>
> ---
> Hello,
>
> Added low/high descriptions and applied to the following git branch.
>
>   git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git for-4.3-unified-base
>
> The branch currently only contains this patch and will stay stable so
> that it can be pulled from.  I kept the base weight as DFL for now.
> If we decide to change it, I'll apply the change on top.
>
> Thanks.
>
>   Documentation/cgroups/unified-hierarchy.txt | 80 ++++++++++++++++++++++++++---
>   include/linux/cgroup.h                      |  9 ++++
>   2 files changed, 81 insertions(+), 8 deletions(-)
>
> diff --git a/Documentation/cgroups/unified-hierarchy.txt b/Documentation/cgroups/unified-hierarchy.txt
> index 86847a7..1ee9caf 100644
> --- a/Documentation/cgroups/unified-hierarchy.txt
> +++ b/Documentation/cgroups/unified-hierarchy.txt
> @@ -23,10 +23,13 @@ CONTENTS
>   5. Other Changes
>     5-1. [Un]populated Notification
>     5-2. Other Core Changes
> -  5-3. Per-Controller Changes
> -    5-3-1. blkio
> -    5-3-2. cpuset
> -    5-3-3. memory
> +  5-3. Controller File Conventions
> +    5-3-1. Format
> +    5-3-2. Control Knobs
> +  5-4. Per-Controller Changes
> +    5-4-1. blkio
> +    5-4-2. cpuset
> +    5-4-3. memory
>   6. Planned Changes
>     6-1. CAP for resource control
>
> @@ -372,14 +375,75 @@ supported and the interface files "release_agent" and
>   - The "cgroup.clone_children" file is removed.
>
>
> -5-3. Per-Controller Changes
> +5-3. Controller File Conventions
>
> -5-3-1. blkio
> +5-3-1. Format
> +
> +In general, all controller files should be in one of the following
> +formats whenever possible.
> +
> +- Values only files
> +
> +  VAL0 VAL1...\n
> +
> +- Flat keyed files
> +
> +  KEY0 VAL0\n
> +  KEY1 VAL1\n
> +  ...
> +
> +- Nested keyed files
> +
> +  KEY0 SUB_KEY0=VAL00 SUB_KEY1=VAL01...
> +  KEY1 SUB_KEY0=VAL10 SUB_KEY1=VAL11...
> +  ...
> +
> +For a writeable file, the format for writing should generally match
> +reading; however, controllers may allow omitting later fields or
> +implement restricted shortcuts for most common use cases.
> +
> +For both flat and nested keyed files, only the values for a single key
> +can be written at a time.  For nested keyed files, the sub key pairs
> +may be specified in any order and not all pairs have to be specified.
> +
> +
> +5-3-2. Control Knobs
> +
> +- Settings for a single feature should generally be implemented in a
> +  single file.
> +
> +- In general, the root cgroup should be exempt from resource control
> +  and thus shouldn't have resource control knobs.
> +
> +- If a controller implements ratio based resource distribution, the
> +  control knob should be named "weight" and have the range [1, 10000]
> +  and 100 should be the default value.  The values are chosen to allow
> +  enough and symmetric bias in both directions while keeping it
> +  intuitive (the default is 100%).
> +
> +- If a controller implements an absolute resource guarantee and/or
> +  limit, the control knobs should be named "min" and "max"
> +  respectively.  If a controller implements best effort resource
> +  gurantee and/or limit, the control knobs should be named "low" and
> +  "high" respectively.
> +
> +  In the above four control files, the special token "max" should be
> +  used to represent upward infinity for both reading and writing.
> +
so, for memory controller, we'll have

(in alphabet order)
memory.failcnt
memory.force_empty  (<= should this be removed ?)
memory.kmem.failcnt
memory.kmem.max
memory.kmem.max_usage
memory.kmem.slabinfo
memory.kmem.tcp.failcnt
memory.kmem.tcp.max
memory.kmem.tcp.max_usage
memory.kmem.tcp.usage
memory.kmem.usage
memory.max
memory.max_usage
memory.move_charge_at_immigrate
memory.numa_stat
memory.oom_control
memory.pressure_level
memory.high
memory.swapiness
memory.usage
memory.use_hierarchy (<= removed)

?
-Kame

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1200486

On Wed 05-08-15 09:39:40, KAMEZAWA Hiroyuki wrote:
[...]
> so, for memory controller, we'll have

We currently have only current, low, high, max and events currently.
All other knobs are either deprecated or waiting for a usecase to emerge
before they get added.

> (in alphabet order)
> memory.failcnt
> memory.force_empty  (<= should this be removed ?)
> memory.kmem.failcnt
> memory.kmem.max
> memory.kmem.max_usage
> memory.kmem.slabinfo
> memory.kmem.tcp.failcnt
> memory.kmem.tcp.max
> memory.kmem.tcp.max_usage
> memory.kmem.tcp.usage
> memory.kmem.usage
> memory.max
> memory.max_usage
> memory.move_charge_at_immigrate
> memory.numa_stat
> memory.oom_control
> memory.pressure_level
> memory.high
> memory.swapiness
> memory.usage
> memory.use_hierarchy (<= removed)

-- 
Michal Hocko
SUSE Labs
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1201395

On 2015/08/05 16:47, Michal Hocko wrote:
> On Wed 05-08-15 09:39:40, KAMEZAWA Hiroyuki wrote:
> [...]
>> so, for memory controller, we'll have
>
> We currently have only current, low, high, max and events currently.
> All other knobs are either deprecated or waiting for a usecase to emerge
> before they get added.
>

Sure. I think following has users.
  - *.stat             - for chekcing health of cgroup ,or for debug
  - *.pressure_level   - for notifying memory pressure
  - *.swappiness       - for adjusting LRU activity per application type.
  - *.oom_control      - for surviving/notifiyng out of memory
                         memcg's oom can be recovered if limit goes up rather than kill.
                         But I know people says this knob is not useful. This will require
                         discussion.

Hm. If we don't want to increase files, NETLINK or systemcall is an another choice of
subsystem specific interface ?

-Kame

>> (in alphabet order)
>> memory.failcnt
>> memory.force_empty  (<= should this be removed ?)
>> memory.kmem.failcnt
>> memory.kmem.max
>> memory.kmem.max_usage
>> memory.kmem.slabinfo
>> memory.kmem.tcp.failcnt
>> memory.kmem.tcp.max
>> memory.kmem.tcp.max_usage
>> memory.kmem.tcp.usage
>> memory.kmem.usage
>> memory.max
>> memory.max_usage
>> memory.move_charge_at_immigrate
>> memory.numa_stat
>> memory.oom_control
>> memory.pressure_level
>> memory.high
>> memory.swapiness
>> memory.usage
>> memory.use_hierarchy (<= removed)
>


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1202982

On Thu 06-08-15 11:30:08, KAMEZAWA Hiroyuki wrote:
[...]
> Sure. I think following has users.
>  - *.stat             - for chekcing health of cgroup ,or for debug

Yes but we want to have something which is closer to meminfo/vmstat IMO

>  - *.pressure_level   - for notifying memory pressure

Notifications are definitely useful I am just not sure this interface is
the right one. We have seen some requests to adjust the interface to get
new semantics (edge vs. level triggered). This should be sorted out
before we expose the knob.

>  - *.swappiness       - for adjusting LRU activity per application type.

Yes, and I wanted to post a patch to export it several times but then I
realized that this should be done only as long as vm.swappiness stays
and it is not deprecated. And more and more I think about swappiness
the less sure I am about it's usefulness. It is not doing much for
quite some time because we are heavily biasing to the pagecache reclaim
and the knob is more and more misleading. It is also not offering what
people might want it to do. E.g. it doesn't allow for preferring swapout
which might be useful when the swap is backed by a really fast storage.
Maybe we will need a new metric here so I wouldn't rush exporting memcg
alternative much.

>  - *.oom_control      - for surviving/notifiyng out of memory
>                         memcg's oom can be recovered if limit goes up rather than kill.

I think it is very much useful - when used wisely. I have seen many
calls for user defined OOM policies but then we have seen those that are
more creative like having the policy maker live in the same memcg which
requires some hacks to prevent from self-deadlocks.
So overall this is very attractive but we might need to think about a
better interface. BPF sounds like a potential way to go. I feel the
memcg and the global approaches should be consistent as much as possible
wrt. API.
-- 
Michal Hocko
SUSE Labs
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1208913

On Fri, Aug 07, 2015 at 08:17:23PM +0200, Michal Hocko wrote:
> On Thu 06-08-15 11:30:08, KAMEZAWA Hiroyuki wrote:
> >  - *.oom_control      - for surviving/notifiyng out of memory
> >                         memcg's oom can be recovered if limit goes up rather than kill.
> 
> I think it is very much useful - when used wisely. I have seen many
> calls for user defined OOM policies but then we have seen those that are
> more creative like having the policy maker live in the same memcg which
> requires some hacks to prevent from self-deadlocks.
> So overall this is very attractive but we might need to think about a
> better interface. BPF sounds like a potential way to go. I feel the
> memcg and the global approaches should be consistent as much as possible
> wrt. API.

I'm not sure I still see a usecase for this.

The whole idea behind memory.high is to give the user the chance to
monitor the group's health and then act upon that. You can freeze the
group if you must, gather information, kill tasks. This is the way to
implement a custom OOM policy.

memory.max on the other hand tells the *kernel* when to OOM, with all
the implications that a kernel OOM has. Don't configure that when you
don't want your tasks killed.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1208905

On Tue, Aug 04, 2015 at 03:31:01PM -0400, Tejun Heo wrote:
> From 6abc8ca19df0078de17dc38340db3002ed489ce7 Mon Sep 17 00:00:00 2001
> From: Tejun Heo <tj@kernel.org>
> Date: Tue, 4 Aug 2015 15:20:55 -0400
> 
> Traditionally, each cgroup controller implemented whatever interface
> it wanted leading to interfaces which are widely inconsistent.
> Examining the requirements of the controllers readily yield that there
> are only a few control schemes shared among all.
> 
> Two major controllers already had to implement new interface for the
> unified hierarchy due to significant structural changes.  Let's take
> the chance to establish common conventions throughout all controllers.
> 
> This patch defines CGROUP_WEIGHT_MIN/DFL/MAX to be used on all weight
> based control knobs and documents the conventions that controllers
> should follow on the unified hierarchy.  Except for io.weight knob,
> all existing unified hierarchy knobs are already compliant.  A
> follow-up patch will update io.weight.
> 
> v2: Added descriptions of min, low and high knobs.
> 
> Signed-off-by: Tejun Heo <tj@kernel.org>
> Acked-by: Johannes Weiner <hannes@cmpxchg.org>
> Cc: Li Zefan <lizefan@huawei.com>
> Cc: Peter Zijlstra <peterz@infradead.org>

Acked-by: Johannes Weiner <hannes@cmpxchg.org>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web