Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1472139 > unrolled thread

Re: [Documentation] State of CPU controller in cgroup v2

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [Documentation] State of CPU controller in cgroup v2 Tejun Heo <tj@kernel.org> - 2016-08-30 00:30 +0200
    Re: [Documentation] State of CPU controller in cgroup v2 Andy Lutomirski <luto@amacapital.net> - 2016-08-31 05:50 +0200
      Re: [Documentation] State of CPU controller in cgroup v2 Tejun Heo <tj@kernel.org> - 2016-08-31 19:40 +0200
        Re: [Documentation] State of CPU controller in cgroup v2 Andy Lutomirski <luto@amacapital.net> - 2016-08-31 21:20 +0200
          Re: [Documentation] State of CPU controller in cgroup v2 Tejun Heo <tj@kernel.org> - 2016-08-31 23:10 +0200
            Re: [Documentation] State of CPU controller in cgroup v2 Andy Lutomirski <luto@amacapital.net> - 2016-08-31 23:50 +0200
              Re: [Documentation] State of CPU controller in cgroup v2 Tejun Heo <tj@kernel.org> - 2016-09-04 00:10 +0200
                Re: [Documentation] State of CPU controller in cgroup v2 Andy Lutomirski <luto@amacapital.net> - 2016-09-05 19:40 +0200
                  Re: [Documentation] State of CPU controller in cgroup v2 Peter Zijlstra <peterz@infradead.org> - 2016-09-06 12:40 +0200
    Re: [Documentation] State of CPU controller in cgroup v2 Andy Lutomirski <luto@amacapital.net> - 2016-08-31 22:00 +0200

#1472139 — Re: [Documentation] State of CPU controller in cgroup v2

Hello, Andy.

Sorry about the delay.  Was kinda overwhelmed with other things.

On Sat, Aug 20, 2016 at 11:45:55AM -0700, Andy Lutomirski wrote:
> > This becomes clear whenever an entity is allocating memory on behalf
> > of someone else - get_user_pages(), khugepaged, swapoff and so on (and
> > likely userfaultfd too).  When a task is trying to add a page to a
> > VMA, the task might not have any relationship with the VMA other than
> > that it's operating on it for someone else.  The page has to be
> > charged to whoever is responsible for the VMA and the only ownership
> > which can be established is the containing mm_struct.
> 
> This surprises me a bit.  If I do access_process_vm(), then I would
> have expected the charge to go the caller, not the mm being accessed.

It does and should go the target mm.  Who faults in a page shouldn't
be the final determinant in the ownership; otherwise, we end up in
situations where the ownership changes due to, for example,
fluctuations in page fault pattern.  It doesn't make semantical sense
either.  If a kthread is doing PIO for a process, why would it get
charged for the memory it's faulting in?

> What happens if a program calls read(2), though?  A page may be
> inserted into page cache on behalf of an address_space without any
> particular mm being involved.  There will usually be a calling task,
> though.

Most faults are synchronous and the faulting thread is a member of the
mm to be charged, so this usually isn't an issue.  I don't think there
are places where we populate an address_space without knowing who it
is for (as opposed / in addition to who the operator is).

> But this is all very memcg-specific.  What about other cgroups?  I/O
> is per-task, right?  Scheduling is definitely per-task.

They aren't separate.  Think about IOs to write out page cache, CPU
cycles spent reclaiming memory or encrypting writeback IOs.  It's fine
to get more granular with specific resources but the semantics gets
messy for cross-resource accounting and control without proper
scoping.

> > Consider the scenario where you have somebody faulting on behalf of a
> > foreign VMA, but the thread who created and is actively using that VMA
> > is in a different cgroup than the process leader.  Who are we going to
> > charge?  All possible answers seem erratic.
> 
> Indeed, and this problem is probably not solvable in practice unless
> you charge all involved cgroups.  But the caller's *mm* is entirely
> irrelevant here, so I don't see how this implies that cgroups need to
> keep tasks in the same process together.  The relevant entities are
> the calling *task* and the target mm, and you're going to be
> hard-pressed to ensure that they belong to the same cgroup, so I think
> you need to be able handle weird cases in which there isn't an
> obviously correct cgroup to charge.

It is an erratic case which is caused by userland interface allowing
non-sensical configuration.  We can accept it as a necessary trade-off
given big enough benefits or unavoidable constraints but it isn't
something to do willy-nilly.

> > For system-level and process-level operations to not step on each
> > other's toes, they need to agree on the granularity boundary -
> > system-level should be able to treat an application hierarchy as a
> > single unit.  A possible solution is allowing rgroup hirearchies to
> > span across process boundaries and implementing cgroup migration
> > operations which treat such hierarchies as a single unit.  I'm not yet
> > sure whether the boundary should be at program groups or rgroups.
> 
> I think that, if the system cgroup manager is moving processes around
> after starting them and execing the final binary, there will be races
> and confusion, and no about of granularity fiddling will fix that.

I don't see how that statement is true.  For example, if you confine
the hierarhcy to in-process, there is proper isolation and whether
system agent migrates the process or not doesn't make any difference
to the internal hierarchy.

> I know nothing about rgroups.  Are they upstream?

It was linked from the original message.

[7]  http://lkml.kernel.org/r/20160105154503.GC5995@mtj.duckdns.org
     [RFD] cgroup: thread granularity support for cpu controller
     Tejun Heo <tj@kernel.org>

[8]  http://lkml.kernel.org/r/1457710888-31182-1-git-send-email-tj@kernel.org
     [PATCHSET RFC cgroup/for-4.6] cgroup, sched: implement resource group and PRIO_RGRP
     Tejun Heo <tj@kernel.org>

[9]  http://lkml.kernel.org/r/20160311160522.GA24046@htj.duckdns.org
     Example program for PRIO_RGRP
     Tejun Heo <tj@kernel.org>

> > These base-system operations are special regardless of cgroup and we
> > already have sometimes crude ways to affect their behaviors where
> > necessary through sysctl knobs, priorities on specific kernel threads
> > and so on.  cgroup doesn't change the situation all that much.  What
> > gets left in the root cgroup usually are the base-system operations
> > which are outside the scope of cgroup resource control in the first
> > place and cgroup resource graph can treat the root as an opaque anchor
> > point.
> 
> This seems to explain why the controllers need to be able to handle
> things being charged to the root cgroup (or to an unidentifiable
> cgroup, anyway).  That isn't quite the same thing as allowing, from an
> ABI point of view, the root cgroup to contain processes and cgroups
> but not allowing other cgroups to do the same thing.  Consider:

The points are 1. we need the root to be a special container anyway
2. allowing it to be special and contain system-wide consumptions
doesn't make the resource graph inconsistent once all non-system-wide
consumptions are put in non-root cgroups, and 3. this is the most
natural way to handle the situation both from implementation and
interface standpoints as it makes non-cgroup configuration a natural
degenerate case of cgroup configuration.

> suppose that systemd (or some competing cgroup manager) is designed to
> run in the root cgroup namespace.  It presumably expects *itself* to
> be in the root cgroup.  Now try to run it using cgroups v2 in a
> non-root namespace.  I don't see how it can possibly work if it the
> hierarchy constraints don't permit it to create sub-cgroups while it's
> still in the root.  In fact, this seems impossible to fix even with
> user code changes.  The manager would need to simultaneously create a
> new child cgroup to contain itself and assign itself to that child
> cgroup, because the intermediate state is illegal.

Please re-read the constraint.  It doesn't prevent any organizational
operations before resource control is enabled.

> I really, really think that cgroup v2 should supply the same
> *interface* inside and outside of a non-root namespace.  If this is

It *does*.  That's what I tried to explain, that it's exactly
isomorhpic once you discount the system-wide consumptions.

Thanks.

-- 
tejun

[toc] | [next] | [standalone]

#1473018

On Mon, Aug 29, 2016 at 3:20 PM, Tejun Heo <tj@kernel.org> wrote:
>> > These base-system operations are special regardless of cgroup and we
>> > already have sometimes crude ways to affect their behaviors where
>> > necessary through sysctl knobs, priorities on specific kernel threads
>> > and so on.  cgroup doesn't change the situation all that much.  What
>> > gets left in the root cgroup usually are the base-system operations
>> > which are outside the scope of cgroup resource control in the first
>> > place and cgroup resource graph can treat the root as an opaque anchor
>> > point.
>>
>> This seems to explain why the controllers need to be able to handle
>> things being charged to the root cgroup (or to an unidentifiable
>> cgroup, anyway).  That isn't quite the same thing as allowing, from an
>> ABI point of view, the root cgroup to contain processes and cgroups
>> but not allowing other cgroups to do the same thing.  Consider:
>
> The points are 1. we need the root to be a special container anyway

But you don't need to let userspace see that.

> 2. allowing it to be special and contain system-wide consumptions
> doesn't make the resource graph inconsistent once all non-system-wide
> consumptions are put in non-root cgroups, and 3. this is the most
> natural way to handle the situation both from implementation and
> interface standpoints as it makes non-cgroup configuration a natural
> degenerate case of cgroup configuration.
>
>> suppose that systemd (or some competing cgroup manager) is designed to
>> run in the root cgroup namespace.  It presumably expects *itself* to
>> be in the root cgroup.  Now try to run it using cgroups v2 in a
>> non-root namespace.  I don't see how it can possibly work if it the
>> hierarchy constraints don't permit it to create sub-cgroups while it's
>> still in the root.  In fact, this seems impossible to fix even with
>> user code changes.  The manager would need to simultaneously create a
>> new child cgroup to contain itself and assign itself to that child
>> cgroup, because the intermediate state is illegal.
>
> Please re-read the constraint.  It doesn't prevent any organizational
> operations before resource control is enabled.
>
>> I really, really think that cgroup v2 should supply the same
>> *interface* inside and outside of a non-root namespace.  If this is
>
> It *does*.  That's what I tried to explain, that it's exactly
> isomorhpic once you discount the system-wide consumptions.
>

I don't think I agree.

Suppose I wrote an init program or a cgroup manager.  I can expect
that init program to be started in the root cgroup.  The program can
be lazy and write +io to /cgroup/cgroup.subtree_control and then
create some new cgroup /cgroup/a and it will work (I just tried it).

Now I run that program in a namespace.  It will not work because it'll
get -EBUSY when it tries to write to cgroup.subtree_control.  (I just
tried this, too, only using cd instead of a namespace.)  So it's *not*
isomorphic.

It *also* won't work (I think) if subtree control is enabled on the
root, but I don't think this is a problem in practice because subtree
control won't be enabled on the namespace root by a sensible cgroup
manager.

--Andy

[toc] | [prev] | [next] | [standalone]

#1473747

Hello, Andy.

On Tue, Aug 30, 2016 at 08:42:20PM -0700, Andy Lutomirski wrote:
> On Mon, Aug 29, 2016 at 3:20 PM, Tejun Heo <tj@kernel.org> wrote:
> >> This seems to explain why the controllers need to be able to handle
> >> things being charged to the root cgroup (or to an unidentifiable
> >> cgroup, anyway).  That isn't quite the same thing as allowing, from an
> >> ABI point of view, the root cgroup to contain processes and cgroups
> >> but not allowing other cgroups to do the same thing.  Consider:
> >
> > The points are 1. we need the root to be a special container anyway
> 
> But you don't need to let userspace see that.

I'm not saying that what cgroup v2 implements is the only solution.
There of course can be other approaches which don't expose this
particular detail to userland.  I was highlighting that there is an
underlying condition to be dealt with and that what cgroup v2
implements is one working solution for it.

It's fine to have, say, aesthetical disgreements on the specifics of
the chosen approach, and, while a bit late, we can still talk about
pros and cons of different possible approaches and make improvements
where it makes sense.  However, this isn't in any way a
make-it-or-break-it issue as you implied before.

> >> I really, really think that cgroup v2 should supply the same
> >> *interface* inside and outside of a non-root namespace.  If this is
> >
> > It *does*.  That's what I tried to explain, that it's exactly
> > isomorhpic once you discount the system-wide consumptions.
> 
> I don't think I agree.
> 
> Suppose I wrote an init program or a cgroup manager.  I can expect
> that init program to be started in the root cgroup.  The program can
> be lazy and write +io to /cgroup/cgroup.subtree_control and then
> create some new cgroup /cgroup/a and it will work (I just tried it).
> 
> Now I run that program in a namespace.  It will not work because it'll
> get -EBUSY when it tries to write to cgroup.subtree_control.  (I just
> tried this, too, only using cd instead of a namespace.)  So it's *not*
> isomorphic.

Yeah, it is possible to shoot yourself in the foot but both
system-scope and namespace-scope can implement the exactly same
behavior - move yourself out of root before enabling resource controls
and get the same expected outcome, which BTW is how systemd behaves
already.

You can say that allowing the possibility of deviation isn't a good
design choice but it is a design choice with other implications - on
how we deal with configurations without cgroup at all, transitioning
from v1, bootstrapping a system and avoiding surprising
userland-visible behaviors (e.g. like creating magic preset cgroups
and silently migrating process there on certain events).

> It *also* won't work (I think) if subtree control is enabled on the
> root, but I don't think this is a problem in practice because subtree
> control won't be enabled on the namespace root by a sensible cgroup
> manager.

Exactly the same thing.  You can shoot yourself in the foot but it's
easy not to.

Thanks.

-- 
tejun

[toc] | [prev] | [next] | [standalone]

#1473798

On Wed, Aug 31, 2016 at 10:32 AM, Tejun Heo <tj@kernel.org> wrote:
> Hello, Andy.
>

>
>> >> I really, really think that cgroup v2 should supply the same
>> >> *interface* inside and outside of a non-root namespace.  If this is
>> >
>> > It *does*.  That's what I tried to explain, that it's exactly
>> > isomorhpic once you discount the system-wide consumptions.
>>
>> I don't think I agree.
>>
>> Suppose I wrote an init program or a cgroup manager.  I can expect
>> that init program to be started in the root cgroup.  The program can
>> be lazy and write +io to /cgroup/cgroup.subtree_control and then
>> create some new cgroup /cgroup/a and it will work (I just tried it).
>>
>> Now I run that program in a namespace.  It will not work because it'll
>> get -EBUSY when it tries to write to cgroup.subtree_control.  (I just
>> tried this, too, only using cd instead of a namespace.)  So it's *not*
>> isomorphic.
>
> Yeah, it is possible to shoot yourself in the foot but both
> system-scope and namespace-scope can implement the exactly same
> behavior - move yourself out of root before enabling resource controls
> and get the same expected outcome, which BTW is how systemd behaves
> already.
>
> You can say that allowing the possibility of deviation isn't a good
> design choice but it is a design choice with other implications - on
> how we deal with configurations without cgroup at all, transitioning
> from v1, bootstrapping a system and avoiding surprising
> userland-visible behaviors (e.g. like creating magic preset cgroups
> and silently migrating process there on certain events).

Are there existing userspace programs that use cgroup2 and enable
subtree control on / when there are processes in /?  If the answer is
no, then I think you should change cgroup2 to just disallow it.  If
the answer is yes, then I think there's a problem and maybe you should
consider a breaking change.  Given that cgroup2 hasn't really launched
on a large scale, it seems worthwhile to get it right.

I don't understand what you're talking about wrt silently migrating
processes.  Are you thinking about usermodehelper?  If so, maybe it
really does make sense to allow (or require?) the cgroup manager to
specify which cgroup these processes end up in.

But, given that all the controllers need to support the current magic
root exception (for genuinely unaccountable things if nothing else),
can you explain what would actually go wrong if you just removed the
restriction entirely?

Also, here's an idea to maybe make PeterZ happier: relax the
restriction a bit per-controller.  Currently (except for /), if you
have subtree control enabled you can't have any processes in the
cgroup.  Could you change this so it only applies to certain
controllers?  If the cpu controller is entirely happy to have
processes and cgroups as siblings, then maybe a cgroup with only cpu
subtree control enabled could allow processes to exist.

>
>> It *also* won't work (I think) if subtree control is enabled on the
>> root, but I don't think this is a problem in practice because subtree
>> control won't be enabled on the namespace root by a sensible cgroup
>> manager.
>
> Exactly the same thing.  You can shoot yourself in the foot but it's
> easy not to.
>

Somewhat off-topic: this appears to be either a bug or a misfeature:

bash-4.3# mkdir foo
bash-4.3# ls foo
cgroup.controllers  cgroup.events  cgroup.procs  cgroup.subtree_control
bash-4.3# mkdir foo/io.max  <-- IMO this shouldn't have worked
bash-4.3# echo +io >cgroup.subtree_control
[   40.470712] cgroup: cgroup_addrm_files: failed to add max, err=-17

Shouldn't cgroups with names that potentially conflict with
kernel-provided dentries be disallowed?

--Andy

[toc] | [prev] | [next] | [standalone]

#1473878

Hello,

On Wed, Aug 31, 2016 at 12:11:58PM -0700, Andy Lutomirski wrote:
> > You can say that allowing the possibility of deviation isn't a good
> > design choice but it is a design choice with other implications - on
> > how we deal with configurations without cgroup at all, transitioning
> > from v1, bootstrapping a system and avoiding surprising
> > userland-visible behaviors (e.g. like creating magic preset cgroups
> > and silently migrating process there on certain events).
> 
> Are there existing userspace programs that use cgroup2 and enable
> subtree control on / when there are processes in /?  If the answer is
> no, then I think you should change cgroup2 to just disallow it.  If
> the answer is yes, then I think there's a problem and maybe you should
> consider a breaking change.  Given that cgroup2 hasn't really launched
> on a large scale, it seems worthwhile to get it right.

Adding the restriction isn't difficult from implementation point of
view and for a system agent which control the boot process
implementing that wouldn't be difficult either but I can't see what
the actual benefits of the extra restriction would be and there are
tangible downsides to doing so.

Consider a use case where the user isn't interested in fully
accounting and dividing up system resources but wants to just cap
resource usage from a subset of workloads.  There is no reason to
require such usages to fully contain all processes in non-root
cgroups.  Furthermore, it's not trivial to migrate all processes out
of root to a sub-cgroup unless the agent is in full control of boot
process.

At least up until this point in discussion, I can't see actual
benefits of adding this restriction and the only reason for pushing it
seems the initial misunderstanding and purism.

> I don't understand what you're talking about wrt silently migrating
> processes.  Are you thinking about usermodehelper?  If so, maybe it
> really does make sense to allow (or require?) the cgroup manager to
> specify which cgroup these processes end up in.

That was from one of the ideas that I was considering way back where
enabling resource control in an intermediate node automatically moves
internal processes to a preset cgroup whether visible or hidden, which
would be another way of addressing the problem.

None of these affects what cgroup v2 can do at all and the only thing
the userland is asked to do under the current scheme is "if you wanna
keep the whole system divided up and use the same mode of operations
across system-scope and namespace-scope move out of root while setting
yourself up, which also happens to be what you have to do inside
namespaces anyway."

> But, given that all the controllers need to support the current magic
> root exception (for genuinely unaccountable things if nothing else),
> can you explain what would actually go wrong if you just removed the
> restriction entirely?

I have, multiple times.  Can you please read 2-1-2 of the document in
the original post and take the discussion from there?

> Also, here's an idea to maybe make PeterZ happier: relax the
> restriction a bit per-controller.  Currently (except for /), if you
> have subtree control enabled you can't have any processes in the
> cgroup.  Could you change this so it only applies to certain
> controllers?  If the cpu controller is entirely happy to have
> processes and cgroups as siblings, then maybe a cgroup with only cpu
> subtree control enabled could allow processes to exist.

The document lists several reasons for not doing this and also that
there is no known real world use case for such configuration.

Please also note that the behavior that you're describing is actually
what rgroup implements.  It makes a lot more sense there because
threads and groups share the same configuration mechanism and it only
has to worry about competition among threads (anonymous consumption is
out of scope for rgroup).

> >> It *also* won't work (I think) if subtree control is enabled on the
> >> root, but I don't think this is a problem in practice because subtree
> >> control won't be enabled on the namespace root by a sensible cgroup
> >> manager.
> >
> > Exactly the same thing.  You can shoot yourself in the foot but it's
> > easy not to.
> 
> Somewhat off-topic: this appears to be either a bug or a misfeature:
> 
> bash-4.3# mkdir foo
> bash-4.3# ls foo
> cgroup.controllers  cgroup.events  cgroup.procs  cgroup.subtree_control
> bash-4.3# mkdir foo/io.max  <-- IMO this shouldn't have worked
> bash-4.3# echo +io >cgroup.subtree_control
> [   40.470712] cgroup: cgroup_addrm_files: failed to add max, err=-17
> 
> Shouldn't cgroups with names that potentially conflict with
> kernel-provided dentries be disallowed?

Yeap, the name collisions suck.  I thought about disallowing all
sub-cgroups which starts with "KNOWN_SUBSYS." but that has a
non-trivial chance of breaking users which were happy before when a
new controller gets added.  But, yeah, we at least should disallow the
known filenames.  Will think more about it.

Thanks.

-- 
tejun

[toc] | [prev] | [next] | [standalone]

#1473902

On Wed, Aug 31, 2016 at 2:07 PM, Tejun Heo <tj@kernel.org> wrote:
> Hello,
>
> On Wed, Aug 31, 2016 at 12:11:58PM -0700, Andy Lutomirski wrote:
>> > You can say that allowing the possibility of deviation isn't a good
>> > design choice but it is a design choice with other implications - on
>> > how we deal with configurations without cgroup at all, transitioning
>> > from v1, bootstrapping a system and avoiding surprising
>> > userland-visible behaviors (e.g. like creating magic preset cgroups
>> > and silently migrating process there on certain events).
>>
>> Are there existing userspace programs that use cgroup2 and enable
>> subtree control on / when there are processes in /?  If the answer is
>> no, then I think you should change cgroup2 to just disallow it.  If
>> the answer is yes, then I think there's a problem and maybe you should
>> consider a breaking change.  Given that cgroup2 hasn't really launched
>> on a large scale, it seems worthwhile to get it right.
>
> Adding the restriction isn't difficult from implementation point of
> view and for a system agent which control the boot process
> implementing that wouldn't be difficult either but I can't see what
> the actual benefits of the extra restriction would be and there are
> tangible downsides to doing so.
>
> Consider a use case where the user isn't interested in fully
> accounting and dividing up system resources but wants to just cap
> resource usage from a subset of workloads.  There is no reason to
> require such usages to fully contain all processes in non-root
> cgroups.  Furthermore, it's not trivial to migrate all processes out
> of root to a sub-cgroup unless the agent is in full control of boot
> process.

Then please also consider exactly the same use case while running in a
container.

I'm a bit frustrated that you're saying that my example failure modes
consist of shooting oneself in the foot and then you go on to come up
with your own examples that have precisely the same problem.

>
>> I don't understand what you're talking about wrt silently migrating
>> processes.  Are you thinking about usermodehelper?  If so, maybe it
>> really does make sense to allow (or require?) the cgroup manager to
>> specify which cgroup these processes end up in.
>
> That was from one of the ideas that I was considering way back where
> enabling resource control in an intermediate node automatically moves
> internal processes to a preset cgroup whether visible or hidden, which
> would be another way of addressing the problem.
>
> None of these affects what cgroup v2 can do at all and the only thing
> the userland is asked to do under the current scheme is "if you wanna
> keep the whole system divided up and use the same mode of operations
> across system-scope and namespace-scope move out of root while setting
> yourself up, which also happens to be what you have to do inside
> namespaces anyway."
>
>> But, given that all the controllers need to support the current magic
>> root exception (for genuinely unaccountable things if nothing else),
>> can you explain what would actually go wrong if you just removed the
>> restriction entirely?
>
> I have, multiple times.  Can you please read 2-1-2 of the document in
> the original post and take the discussion from there?

I've read it multiple times, and I don't see any explanation that's
consistent with the fact that you are exempting the root cgroup from
this constraint.  If the constraint were really critical to everything
working, then I would expect the root cgroup to have exactly the same
problem.  This makes me think that either something nasty is being
fudged for the root cgroup or that the constraint isn't actually so
important after all.  The only thing on point I can find is:

> Root cgroup is exempt from this constraint, which is in line with
> how root cgroup is handled in general - it's excluded from cgroup
> resource accounting and control.

and that's not very helpful.

>
>> Also, here's an idea to maybe make PeterZ happier: relax the
>> restriction a bit per-controller.  Currently (except for /), if you
>> have subtree control enabled you can't have any processes in the
>> cgroup.  Could you change this so it only applies to certain
>> controllers?  If the cpu controller is entirely happy to have
>> processes and cgroups as siblings, then maybe a cgroup with only cpu
>> subtree control enabled could allow processes to exist.
>
> The document lists several reasons for not doing this and also that
> there is no known real world use case for such configuration.

My company's production workload would map quite nicely to this
relaxed model.  I have quite a few processes each with several
threads.  Some of those threads get some CPUs, some get other CPUs,
and they vary in what shares of what CPUs they get.  To be clear,
there is not a hierarchy of resource usage that's compatible with the
process hierarchy.  Multiple processes have threads that should be
grouped in a different place in the hierarchy than other threads.
Concretely, I have processes A and B with threads A1, A2, B1, and B2.
(And many more, but this is enough to get the point across.)  The
natural grouping is:

Group 1: A1 and B1
Group 2: A2
Group 3: B2

This cannot be expressed with rgroup or with cgroup2.  cgroup1 has no
problem with it.  If I were using memcg, I would want to have a memcg
hierarchy that was incompatible with the hierarchy above, so I
actually find the cgroup2 insistence on a unified hierarchy to be a
bit annoying, but I at least understand the motivation behind the
unified hierarchy.

And I don't care that the system controller can't atomically move this
whole mess around.  I'm currently running without systemd, so I don't
*have* a system controller.  If I end up migrating to systemd, I'll
probably put this whole pile into its own slice and manage it
manually.

>
>> >> It *also* won't work (I think) if subtree control is enabled on the
>> >> root, but I don't think this is a problem in practice because subtree
>> >> control won't be enabled on the namespace root by a sensible cgroup
>> >> manager.
>> >
>> > Exactly the same thing.  You can shoot yourself in the foot but it's
>> > easy not to.
>>
>> Somewhat off-topic: this appears to be either a bug or a misfeature:
>>
>> bash-4.3# mkdir foo
>> bash-4.3# ls foo
>> cgroup.controllers  cgroup.events  cgroup.procs  cgroup.subtree_control
>> bash-4.3# mkdir foo/io.max  <-- IMO this shouldn't have worked
>> bash-4.3# echo +io >cgroup.subtree_control
>> [   40.470712] cgroup: cgroup_addrm_files: failed to add max, err=-17
>>
>> Shouldn't cgroups with names that potentially conflict with
>> kernel-provided dentries be disallowed?
>
> Yeap, the name collisions suck.  I thought about disallowing all
> sub-cgroups which starts with "KNOWN_SUBSYS." but that has a
> non-trivial chance of breaking users which were happy before when a
> new controller gets added.  But, yeah, we at least should disallow the
> known filenames.  Will think more about it.

How about disallowing names that contain a '.'?

--Andy

[toc] | [prev] | [next] | [standalone]

#1475797

Hello, Andy.

On Wed, Aug 31, 2016 at 02:46:20PM -0700, Andy Lutomirski wrote:
> > Consider a use case where the user isn't interested in fully
> > accounting and dividing up system resources but wants to just cap
> > resource usage from a subset of workloads.  There is no reason to
> > require such usages to fully contain all processes in non-root
> > cgroups.  Furthermore, it's not trivial to migrate all processes out
> > of root to a sub-cgroup unless the agent is in full control of boot
> > process.
> 
> Then please also consider exactly the same use case while running in a
> container.
> 
> I'm a bit frustrated that you're saying that my example failure modes
> consist of shooting oneself in the foot and then you go on to come up
> with your own examples that have precisely the same problem.

You have a point, which is

  The system-root and namespace-roots are not symmetric.

and that's a valid concern.  Here's why the system-root is special.

* A system has entities and resource consumptions which can only be
  attributed to the "system".  The system-root is the natural place to
  put them.  The system-root has stuff no other cgroups, not even
  namespace-roots, have.  It's a unique situation.

* The need to bypass most cgroup related overhead when not in use.
  The system-root is there whether cgroup is actally in use or not and
  thus can not impose noticeable overhead.  It has to make sense for
  both resource-controlled systems as well as ones that aren't.
  Again, no other group has these requirements.

  Note that this means that all controllers should be able to and
  already allow uncontained consumptions in the system-root.  I'll
  come back to this later.

Now, due to the various issues with direct competition between
processes and cgroups, cgroup v2 disallows resource control across
them (the no-internal-tasks restriction); however, cgroup v2 currently
doesn't apply the restriction to the system-root.  Here are the
reasons.

* It doesn't bring any practical benefits in terms of implementation.
  As noted above, all controllers already have to allow uncontained
  consumptions in the system-root and that's the only attribute
  required for the exemption.

* It doesn't bring any practical benefits in terms of capability.
  Userland can trivially handle the system-root and namespace-roots in
  a symmetrical manner.

* It's an unncessary inconvenience, especially for cases where the
  cgroup agent isn't in control of boot, for partial usage cases, or
  just for playing with it.

You say that I'm ignoring the same use case for namespace-scope but
namespace-roots don't have the same hybrid function for partial and
uncontrolled systems, so it's not clear why there even NEEDS to be
strict symmetry.

On this subject, your only actual point is that there is an asymmetry
and that's bothersome.  I've been trying to explain why the special
case doesn't actually get in the way in terms of implementation or
capability and is actually beneficial.  Instead of engaging in the
actual discussion, you're constantly coming up with different ways of
saying "it's not symmetric".

The system-root and namespace-roots aren't equivalent.  There are a
lot of parallels between system-root and namescope-root but they
aren't the same thing (e.g. bootstrapping a namespace is a less
complicated and more malleable process).  The system-root is not even
a fully qualified node of the resource graph.

It's easy and understandable to get hangups on asymmetries or
exemptions like this, but they also often are acceptable trade-offs.
It's really frustrating to see you first getting hung up on "this must
be wrong" and even after explanations repeating the same thing just in
different ways.

If there is something fundamentally wrong with it, sure, let's fix it,
but what's actually broken?

> > I have, multiple times.  Can you please read 2-1-2 of the document in
> > the original post and take the discussion from there?
> 
> I've read it multiple times, and I don't see any explanation that's
> consistent with the fact that you are exempting the root cgroup from
> this constraint.  If the constraint were really critical to everything
> working, then I would expect the root cgroup to have exactly the same
> problem.  This makes me think that either something nasty is being
> fudged for the root cgroup or that the constraint isn't actually so
> important after all.  The only thing on point I can find is:
> 
> > Root cgroup is exempt from this constraint, which is in line with
> > how root cgroup is handled in general - it's excluded from cgroup
> > resource accounting and control.
> 
> and that's not very helpful.

My apologies.  I somehow thought that was part of the documentation.
Will update it later, but here's an excerpt from my earlier response.

  Having a special case doesn't necessarily get in the way of
  benefiting from a set of general rules.  The root cgroup is
  inherently special as it has to be the catch-all scope for entities
  and resource consumptions which can't be tied to any specific
  consumer - irq handling, packet rx, journal writes, memory reclaim
  from global memory pressure and so on.  None of sub-cgroups have to
  worry about them.

  These base-system operations are special regardless of cgroup and we
  already have sometimes crude ways to affect their behaviors where
  necessary through sysctl knobs, priorities on specific kernel
  threads and so on.  cgroup doesn't change the situation all that
  much.  What gets left in the root cgroup usually are the base-system
  operations which are outside the scope of cgroup resource control in
  the first place and cgroup resource graph can treat the root as an
  opaque anchor point.

  There can be other ways to deal with the issue; however, treating
  root cgroup this way has the big advantage of minimizing the gap
  between configurations without and with cgroups both in terms of
  mental model and implementation.

  Hopefully, the case of a namespace root is clear now.  If it's gonna
  have a sub-hierarchy, it itself can't contain processes but the
  system root just contains base-system entities and resources which a
  namespace root doesn't have to worry about.  Ignoring base-system
  stuff, a namespace root is topologically in the same position as the
  system root in the cgroup resource graph.

Maybe this wasn't as clear as I thought it was.  I hope the earlier
part of this message is enough of a clarification.

> >> Also, here's an idea to maybe make PeterZ happier: relax the
> >> restriction a bit per-controller.  Currently (except for /), if you
> >> have subtree control enabled you can't have any processes in the
> >> cgroup.  Could you change this so it only applies to certain
> >> controllers?  If the cpu controller is entirely happy to have
> >> processes and cgroups as siblings, then maybe a cgroup with only cpu
> >> subtree control enabled could allow processes to exist.
> >
> > The document lists several reasons for not doing this and also that
> > there is no known real world use case for such configuration.

So, up until this point, we were talking about no-internal-tasks
constraint.

> My company's production workload would map quite nicely to this
> relaxed model.  I have quite a few processes each with several
> threads.  Some of those threads get some CPUs, some get other CPUs,
> and they vary in what shares of what CPUs they get.  To be clear,
> there is not a hierarchy of resource usage that's compatible with the
> process hierarchy.  Multiple processes have threads that should be
> grouped in a different place in the hierarchy than other threads.
> Concretely, I have processes A and B with threads A1, A2, B1, and B2.
> (And many more, but this is enough to get the point across.)  The
> natural grouping is:
> 
> Group 1: A1 and B1
> Group 2: A2
> Group 3: B2

And now you're talking about process granularity.

> This cannot be expressed with rgroup or with cgroup2.  cgroup1 has no
> problem with it.  If I were using memcg, I would want to have a memcg
> hierarchy that was incompatible with the hierarchy above, so I
> actually find the cgroup2 insistence on a unified hierarchy to be a
> bit annoying, but I at least understand the motivation behind the
> unified hierarchy.
> 
> And I don't care that the system controller can't atomically move this
> whole mess around.  I'm currently running without systemd, so I don't

I do.  It's a horrible userland API to expose to individual
applications if the organization that a given application expects can
be disturbed by system operations.  Imagine how this would be
documented - "if this operation races with system operation, it may
return -ENOENT.  Repeating the path lookup might make the operation
succeed again."

> *have* a system controller.  If I end up migrating to systemd, I'll
> probably put this whole pile into its own slice and manage it
> manually.

Yeah, systemd has delegation feature for cases like that which we
depend on too.

As for your example, who performs the cgroup setup and configuration,
the application itself or an external entity?  If an external entity,
how does it know which thread is what?

And, as for rgroup not covering it, would extending rgroup to cover
multi-process cases be enough or are there more fundamental issues?

> > Yeap, the name collisions suck.  I thought about disallowing all
> > sub-cgroups which starts with "KNOWN_SUBSYS." but that has a
> > non-trivial chance of breaking users which were happy before when a
> > new controller gets added.  But, yeah, we at least should disallow the
> > known filenames.  Will think more about it.
> 
> How about disallowing names that contain a '.'?

That's guaranteed to break things left and right, and, given how
departed it is from what has been all along including v1, it'd be an
actually gratuitous painful change.  While name collisions is a nasty
possibility, it seldom is a practical problem as most use naming
schemes which are unlikely to actually collide.  Even "$SUBSYS." is
likely too broad.  Most cures seem worse than the disease here.

Thanks.

-- 
tejun

[toc] | [prev] | [next] | [standalone]

#1476832

On Sat, Sep 3, 2016 at 3:05 PM, Tejun Heo <tj@kernel.org> wrote:
> Hello, Andy.
>
> On Wed, Aug 31, 2016 at 02:46:20PM -0700, Andy Lutomirski wrote:
>> > Consider a use case where the user isn't interested in fully
>> > accounting and dividing up system resources but wants to just cap
>> > resource usage from a subset of workloads.  There is no reason to
>> > require such usages to fully contain all processes in non-root
>> > cgroups.  Furthermore, it's not trivial to migrate all processes out
>> > of root to a sub-cgroup unless the agent is in full control of boot
>> > process.
>>
>> Then please also consider exactly the same use case while running in a
>> container.
>>
>> I'm a bit frustrated that you're saying that my example failure modes
>> consist of shooting oneself in the foot and then you go on to come up
>> with your own examples that have precisely the same problem.
>
> You have a point, which is
>
>   The system-root and namespace-roots are not symmetric.
>
> and that's a valid concern.  Here's why the system-root is special.
>

[...]

>
> Now, due to the various issues with direct competition between
> processes and cgroups, cgroup v2 disallows resource control across
> them (the no-internal-tasks restriction); however, cgroup v2 currently
> doesn't apply the restriction to the system-root.  Here are the
> reasons.
>
> * It doesn't bring any practical benefits in terms of implementation.
>   As noted above, all controllers already have to allow uncontained
>   consumptions in the system-root and that's the only attribute
>   required for the exemption.
>
> * It doesn't bring any practical benefits in terms of capability.
>   Userland can trivially handle the system-root and namespace-roots in
>   a symmetrical manner.

Your idea of "trivially" doesn't match mine.  You gave a use case in
which userspace might take advantage of root being special.  If
userspace does that, then that userspace cannot be run in a container.
This could be a problem for real users.  Sure, "don't do that" is a
*valid* answer, but it's not a very helpful answer.

>
> * It's an unncessary inconvenience, especially for cases where the
>   cgroup agent isn't in control of boot, for partial usage cases, or
>   just for playing with it.
>
> You say that I'm ignoring the same use case for namespace-scope but
> namespace-roots don't have the same hybrid function for partial and
> uncontrolled systems, so it's not clear why there even NEEDS to be
> strict symmetry.

I think their functions are much closer than you think they are.  I
want a whole Linux distro to be able to run in a container.  This
means that useful things people do in a distro or initramfs or
whatever should just work if containerized.

>
> It's easy and understandable to get hangups on asymmetries or
> exemptions like this, but they also often are acceptable trade-offs.
> It's really frustrating to see you first getting hung up on "this must
> be wrong" and even after explanations repeating the same thing just in
> different ways.
>
> If there is something fundamentally wrong with it, sure, let's fix it,
> but what's actually broken?

I'm not saying it's fundamentally wrong.  I'm saying it's a design
that has a big wart, and that wart is unfortunate, and after thinking
a bit, I'm starting to agree with PeterZ that this is problematic.  It
also seems fixable: the constraint could be relaxed.

>> >> Also, here's an idea to maybe make PeterZ happier: relax the
>> >> restriction a bit per-controller.  Currently (except for /), if you
>> >> have subtree control enabled you can't have any processes in the
>> >> cgroup.  Could you change this so it only applies to certain
>> >> controllers?  If the cpu controller is entirely happy to have
>> >> processes and cgroups as siblings, then maybe a cgroup with only cpu
>> >> subtree control enabled could allow processes to exist.
>> >
>> > The document lists several reasons for not doing this and also that
>> > there is no known real world use case for such configuration.
>
> So, up until this point, we were talking about no-internal-tasks
> constraint.

Isn't this the same thing?  IIUC the constraint in question is that,
if a non-root cgroup has subtree control on, then it can't have
processes in it.  This is the no-internal-tasks constraint, right?

And I still think that, at least for cpu, nothing at all goes wrong if
you allow processes to exist in cgroups that have cpu set in
subtree-control.

----- begin talking about process granularity -----

>
>> My company's production workload would map quite nicely to this
>> relaxed model.  I have quite a few processes each with several
>> threads.  Some of those threads get some CPUs, some get other CPUs,
>> and they vary in what shares of what CPUs they get.  To be clear,
>> there is not a hierarchy of resource usage that's compatible with the
>> process hierarchy.  Multiple processes have threads that should be
>> grouped in a different place in the hierarchy than other threads.
>> Concretely, I have processes A and B with threads A1, A2, B1, and B2.
>> (And many more, but this is enough to get the point across.)  The
>> natural grouping is:
>>
>> Group 1: A1 and B1
>> Group 2: A2
>> Group 3: B2
>
> And now you're talking about process granularity.

Yes.

>
>> This cannot be expressed with rgroup or with cgroup2.  cgroup1 has no
>> problem with it.  If I were using memcg, I would want to have a memcg
>> hierarchy that was incompatible with the hierarchy above, so I
>> actually find the cgroup2 insistence on a unified hierarchy to be a
>> bit annoying, but I at least understand the motivation behind the
>> unified hierarchy.
>>
>> And I don't care that the system controller can't atomically move this
>> whole mess around.  I'm currently running without systemd, so I don't
>
> I do.  It's a horrible userland API to expose to individual
> applications if the organization that a given application expects can
> be disturbed by system operations.  Imagine how this would be
> documented - "if this operation races with system operation, it may
> return -ENOENT.  Repeating the path lookup might make the operation
> succeed again."

It could be made to work without races, though, with minimal (or even
no) ABI change.  The managed program could grab an fd pointing to its
cgroup.  Then it would use openat, etc for all operations.  As long as
'mv /cgroup/a/b /cgroup/c/" didn't cause that fd to stop working,
we're fine.

Note that this pretty much has to work if cgroup namespaces are to
allow rearrangement of the hierarchy -- '/cgroup/' from inside the
namespace has to remain valid at all times

Obviously this only works if the cgroup in question doesn't itself get
destroyed, but having an internal hierarchy is a bit nonsensical if
the application shares a cgroup with another application, so that
shouldn't be a problem in practice.

In fact, ISTM that allowing applications to manage cgroup
sub-hierarchies has almost exactly the same set of constraints as
allowing namespaced cgroup managers to work.  In a container, the
outer manager manages where the container lives and the container
manages its own hierarchy.  Why can't fancy cgroup-aware applications
work exactly the same way?

>
>> *have* a system controller.  If I end up migrating to systemd, I'll
>> probably put this whole pile into its own slice and manage it
>> manually.
>
> Yeah, systemd has delegation feature for cases like that which we
> depend on too.
>
> As for your example, who performs the cgroup setup and configuration,
> the application itself or an external entity?  If an external entity,
> how does it know which thread is what?

In my case, it would be a little script that reads a config file that
knows all kinds of internal information about the application and its
threads.

>
> And, as for rgroup not covering it, would extending rgroup to cover
> multi-process cases be enough or are there more fundamental issues?

Maybe, as long as the configuration could actually be created -- IIUC
the current rgroup proposal requires that the hierarchy of groups
matches the hierarchy implied by clone(), which isn't going to happen
in my case.

But, given that this fancy-cgroup-aware-multiprocess-application case
looks so much like cgroup-using container, ISTM you could solve the
problem completely by just allowing tasks to be split out by users who
want to do it.  (Obviously those users will get funny results if they
try to do this to memcg.  "Don't do that" seems fine here.)  I don't
expect the race condition issues you're worried about to happen in
practice.  Certainly not in my case, since I control the entire
system.

[toc] | [prev] | [next] | [standalone]

#1477268

On Mon, Sep 05, 2016 at 10:37:55AM -0700, Andy Lutomirski wrote:
> And I still think that, at least for cpu, nothing at all goes wrong if
> you allow processes to exist in cgroups that have cpu set in
> subtree-control.

cpu, cpuset, perf, cpuacct (although we all agree that really should be
part of cpu), pid, and possibly freezer (but I think we all agree
freezer is 'broken').

That's roughly half the controllers out there.

They all work on tasks, and should therefore have no problems what so
ever to allow the full hierarchy without silly exceptions and
constraints.



The fundamental problem is that we have 2 different types of
controllers, on the one hand these controllers above, that work on tasks
and form groups of them and build up from that. Lets call them
task-controllers.

On the other hand we have controllers like memcg which take the 'system'
as a whole and shrink it down into smaller bits. Lets call these
system-controllers.


They are fundamentally at odds with capabilities, simply because of the
granularity they can work on.

Merging the two into a common hierarchy is a useful concept for
containerization, no argument on that, esp. when also coupled with
namespaces and the like.


However, where I object _most_ strongly is having this one use dominate
and destroy the capabilities (which are in use) of the task-controllers.


> > I do.  It's a horrible userland API to expose to individual
> > applications if the organization that a given application expects can
> > be disturbed by system operations.  Imagine how this would be
> > documented - "if this operation races with system operation, it may
> > return -ENOENT.  Repeating the path lookup might make the operation
> > succeed again."
> 
> It could be made to work without races, though, with minimal (or even
> no) ABI change.  The managed program could grab an fd pointing to its
> cgroup.  Then it would use openat, etc for all operations.  As long as
> 'mv /cgroup/a/b /cgroup/c/" didn't cause that fd to stop working,
> we're fine.

I've mentioned openat() and related APIs several times, but so far never
got good reasons why that wouldn't work.



Also note that in order to partition the cpus with cpusets, you're
required to generate a disjoint hierarchy (that is, one where the
(common) parent is 'disabled' and the children have no overlap).

This is rather fundamental to partitioning, that by its very nature
requires separation.

The result is that if you want to place your RT threads (consider an
application that consists of RT and !RT parts) in a different partition
there is no common parent you can place the process in.


cgroup-v2, by placing the system style controllers first and foremost,
completely renders that scenario impossible. Note also that any proposed
rgroup would not work for this, since that, per design, is a subtree,
and therefore not disjoint.


So my objection to the whole cgroup-v2 model and implementation stems
from the fact that it purports to be a 'better' and 'improved' system,
while in actuality it neuters and destroys a lot of useful usecases.

It completely disregards all task-controllers and labels their use-cases
as irrelevant.

[toc] | [prev] | [next] | [standalone]

#1473817

I'm replying separately to keep the two issues in separate emails.

On Mon, Aug 29, 2016 at 3:20 PM, Tejun Heo <tj@kernel.org> wrote:
> Hello, Andy.
>
> Sorry about the delay.  Was kinda overwhelmed with other things.
>
> On Sat, Aug 20, 2016 at 11:45:55AM -0700, Andy Lutomirski wrote:
>> > This becomes clear whenever an entity is allocating memory on behalf
>> > of someone else - get_user_pages(), khugepaged, swapoff and so on (and
>> > likely userfaultfd too).  When a task is trying to add a page to a
>> > VMA, the task might not have any relationship with the VMA other than
>> > that it's operating on it for someone else.  The page has to be
>> > charged to whoever is responsible for the VMA and the only ownership
>> > which can be established is the containing mm_struct.
>>
>> This surprises me a bit.  If I do access_process_vm(), then I would
>> have expected the charge to go the caller, not the mm being accessed.
>
> It does and should go the target mm.  Who faults in a page shouldn't
> be the final determinant in the ownership; otherwise, we end up in
> situations where the ownership changes due to, for example,
> fluctuations in page fault pattern.  It doesn't make semantical sense
> either.  If a kthread is doing PIO for a process, why would it get
> charged for the memory it's faulting in?

OK, that makes sense.  Although, given that cgroup1 allows tasks in
the same processes to be split up, how does this work in cgroup1?  Do
you just pick the mm associated with the thread group leader?  If so,
why can't cgroup2 do the same thing?

But even this is at best a vague approximation.  If you have
MAP_SHARED mappings (libc.so, for example), then the cgroup you charge
it to is more or less arbitrary.

>
>> What happens if a program calls read(2), though?  A page may be
>> inserted into page cache on behalf of an address_space without any
>> particular mm being involved.  There will usually be a calling task,
>> though.
>
> Most faults are synchronous and the faulting thread is a member of the
> mm to be charged, so this usually isn't an issue.  I don't think there
> are places where we populate an address_space without knowing who it
> is for (as opposed / in addition to who the operator is).

True, but there's no *mm* involved in any fundamental sense.  You can
look at the task and find the task's mm (or actually the task's thread
group leader, since cgroup2 doesn't literally map mms to cgroups), but
that seems to me to be a pretty poor reason to argue that tasks should
have to be kept together.

>
>> But this is all very memcg-specific.  What about other cgroups?  I/O
>> is per-task, right?  Scheduling is definitely per-task.
>
> They aren't separate.  Think about IOs to write out page cache, CPU
> cycles spent reclaiming memory or encrypting writeback IOs.  It's fine
> to get more granular with specific resources but the semantics gets
> messy for cross-resource accounting and control without proper
> scoping.

Page cache doesn't belong to a a specific mm.  Memory reclaim only has
an mm associated if the memory being reclaimed belongs cleanly to an
mm.  Encrypting writeback (I assume you mean the cpu usage) is just
like page cache writeback IO -- there's no specific mm involved in
general.

>
>> > Consider the scenario where you have somebody faulting on behalf of a
>> > foreign VMA, but the thread who created and is actively using that VMA
>> > is in a different cgroup than the process leader.  Who are we going to
>> > charge?  All possible answers seem erratic.
>>
>> Indeed, and this problem is probably not solvable in practice unless
>> you charge all involved cgroups.  But the caller's *mm* is entirely
>> irrelevant here, so I don't see how this implies that cgroups need to
>> keep tasks in the same process together.  The relevant entities are
>> the calling *task* and the target mm, and you're going to be
>> hard-pressed to ensure that they belong to the same cgroup, so I think
>> you need to be able handle weird cases in which there isn't an
>> obviously correct cgroup to charge.
>
> It is an erratic case which is caused by userland interface allowing
> non-sensical configuration.  We can accept it as a necessary trade-off
> given big enough benefits or unavoidable constraints but it isn't
> something to do willy-nilly.
>
>> > For system-level and process-level operations to not step on each
>> > other's toes, they need to agree on the granularity boundary -
>> > system-level should be able to treat an application hierarchy as a
>> > single unit.  A possible solution is allowing rgroup hirearchies to
>> > span across process boundaries and implementing cgroup migration
>> > operations which treat such hierarchies as a single unit.  I'm not yet
>> > sure whether the boundary should be at program groups or rgroups.
>>
>> I think that, if the system cgroup manager is moving processes around
>> after starting them and execing the final binary, there will be races
>> and confusion, and no about of granularity fiddling will fix that.
>
> I don't see how that statement is true.  For example, if you confine
> the hierarhcy to in-process, there is proper isolation and whether
> system agent migrates the process or not doesn't make any difference
> to the internal hierarchy.

But hierarchy isn't always per process.  Some real-world services have
threads and subprocesses.

>
>> I know nothing about rgroups.  Are they upstream?
>
> It was linked from the original message.
>
> [7]  http://lkml.kernel.org/r/20160105154503.GC5995@mtj.duckdns.org
>      [RFD] cgroup: thread granularity support for cpu controller
>      Tejun Heo <tj@kernel.org>

I can see two issues here:

1. You're allowing groups and tasks to be siblings.  If you're okay
allowing that for rgroups, why not allow it for cgroup2 on the same
set of controllers?

2. It looks impossible to fork and keep a child in the same group as
one of your non-leader threads.

I think I'm starting to agree with PeterZ here.  Why not just make
cgroup2 more flexible?

--Andy

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web