Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1291285 > unrolled thread

Re: [PATCH 1/7] mm: memcontrol: charge swap to cgroup2

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCH 1/7] mm: memcontrol: charge swap to cgroup2 Michal Hocko <mhocko@kernel.org> - 2015-12-14 16:40 +0100
    Re: [PATCH 1/7] mm: memcontrol: charge swap to cgroup2 Johannes Weiner <hannes@cmpxchg.org> - 2015-12-14 16:50 +0100
    Re: [PATCH 1/7] mm: memcontrol: charge swap to cgroup2 Vladimir Davydov <vdavydov@virtuozzo.com> - 2015-12-14 20:50 +0100
      Re: [PATCH 1/7] mm: memcontrol: charge swap to cgroup2 One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> - 2015-12-14 21:00 +0100
      Re: [PATCH 1/7] mm: memcontrol: charge swap to cgroup2 Kamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> - 2015-12-15 04:30 +0100
        Re: [PATCH 1/7] mm: memcontrol: charge swap to cgroup2 Vladimir Davydov <vdavydov@virtuozzo.com> - 2015-12-15 12:10 +0100
          Re: [PATCH 1/7] mm: memcontrol: charge swap to cgroup2 Kamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> - 2015-12-16 03:50 +0100
        Re: [PATCH 1/7] mm: memcontrol: charge swap to cgroup2 Johannes Weiner <hannes@cmpxchg.org> - 2015-12-15 16:00 +0100
          Re: [PATCH 1/7] mm: memcontrol: charge swap to cgroup2 Kamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> - 2015-12-16 04:20 +0100
            Re: [PATCH 1/7] mm: memcontrol: charge swap to cgroup2 Johannes Weiner <hannes@cmpxchg.org> - 2015-12-16 12:10 +0100
              Re: [PATCH 1/7] mm: memcontrol: charge swap to cgroup2 Kamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> - 2015-12-17 03:50 +0100
                Re: [PATCH 1/7] mm: memcontrol: charge swap to cgroup2 Johannes Weiner <hannes@cmpxchg.org> - 2015-12-17 04:40 +0100
                  Re: [PATCH 1/7] mm: memcontrol: charge swap to cgroup2 Kamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> - 2015-12-17 05:40 +0100
      Re: [PATCH 1/7] mm: memcontrol: charge swap to cgroup2 Michal Hocko <mhocko@kernel.org> - 2015-12-15 18:30 +0100
        Re: [PATCH 1/7] mm: memcontrol: charge swap to cgroup2 Johannes Weiner <hannes@cmpxchg.org> - 2015-12-15 21:30 +0100
        Re: [PATCH 1/7] mm: memcontrol: charge swap to cgroup2 Kamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> - 2015-12-16 05:00 +0100
    Re: [PATCH 1/7] mm: memcontrol: charge swap to cgroup2 Kamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> - 2015-12-15 04:30 +0100
      Re: [PATCH 1/7] mm: memcontrol: charge swap to cgroup2 Vladimir Davydov <vdavydov@virtuozzo.com> - 2015-12-15 09:40 +0100
        Re: [PATCH 1/7] mm: memcontrol: charge swap to cgroup2 Kamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> - 2015-12-15 10:40 +0100

#1291285 — Re: [PATCH 1/7] mm: memcontrol: charge swap to cgroup2

On Thu 10-12-15 14:39:14, Vladimir Davydov wrote:
> In the legacy hierarchy we charge memsw, which is dubious, because:
> 
>  - memsw.limit must be >= memory.limit, so it is impossible to limit
>    swap usage less than memory usage. Taking into account the fact that
>    the primary limiting mechanism in the unified hierarchy is
>    memory.high while memory.limit is either left unset or set to a very
>    large value, moving memsw.limit knob to the unified hierarchy would
>    effectively make it impossible to limit swap usage according to the
>    user preference.
> 
>  - memsw.usage != memory.usage + swap.usage, because a page occupying
>    both swap entry and a swap cache page is charged only once to memsw
>    counter. As a result, it is possible to effectively eat up to
>    memory.limit of memory pages *and* memsw.limit of swap entries, which
>    looks unexpected.
> 
> That said, we should provide a different swap limiting mechanism for
> cgroup2.
> This patch adds mem_cgroup->swap counter, which charges the actual
> number of swap entries used by a cgroup. It is only charged in the
> unified hierarchy, while the legacy hierarchy memsw logic is left
> intact.

I agree that the previous semantic was awkward. The problem I can see
with this approach is that once the swap limit is reached the anon
memory pressure might spill over to other and unrelated memcgs during
the global memory pressure. I guess this is what Kame referred to as
anon would become mlocked basically. This would be even more of an issue
with resource delegation to sub-hierarchies because nobody will prevent
setting the swap amount to a small value and use that as an anon memory
protection.

I guess this was the reason why this approach hasn't been chosen before
but I think we can come up with a way to stop the run away consumption
even when the swap is accounted separately. All of them are quite nasty
but let me try.

We could allow charges to fail even for the high limit if the excess is
way above the amount of reclaimable memory in the given memcg/hierarchy.
A runaway load would be stopped before it can cause a considerable
damage outside of its hierarchy this way even when the swap limit
is configured small.
Now that goes against the high limit semantic which should only throttle
the consumer and shouldn't cause any functional failures but maybe this
is acceptable for the overall system stability. An alternative would
be to throttle in the high limit reclaim context proportionally to
the excess. This is normally done by the reclaim itself but with no
reclaimable memory this wouldn't work that way.

Another option would be to ignore the swap limit during the global
reclaim. This wouldn't stop the runaway loads but they would at least
see their fair share of the reclaim. The swap excess could be then used
as a "handicap" for a more aggressive throttling during high limit reclaim
or to trigger hard limit sooner.

Or we could teach the global OOM killer to select abusive anon memory
users with restricted swap. That would require to iterate through all
memcgs and checks whether their anon consumption is in a large excess to
their swap limit and fallback to the memcg OOM victim selection if that
is the case. This adds more complexity to the OOM killer path so I am
not sure this is generally acceptable, though.

My question now is. Is the knob usable/useful even without additional
heuristics? Do we want to protect swap space so rigidly that a swap
limited memcg can cause bigger problems than without the swap limit
globally?

> The swap usage can be monitored using new memory.swap.current file and
> limited using memory.swap.max.
> 
> Signed-off-by: Vladimir Davydov <vdavydov@virtuozzo.com>
> ---
>  include/linux/memcontrol.h |   1 +
>  include/linux/swap.h       |   5 ++
>  mm/memcontrol.c            | 123 +++++++++++++++++++++++++++++++++++++++++----
>  mm/shmem.c                 |   4 ++
>  mm/swap_state.c            |   5 ++
>  5 files changed, 129 insertions(+), 9 deletions(-)

[...]
-- 
Michal Hocko
SUSE Labs
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1291297

On Mon, Dec 14, 2015 at 04:30:37PM +0100, Michal Hocko wrote:
> On Thu 10-12-15 14:39:14, Vladimir Davydov wrote:
> > In the legacy hierarchy we charge memsw, which is dubious, because:
> > 
> >  - memsw.limit must be >= memory.limit, so it is impossible to limit
> >    swap usage less than memory usage. Taking into account the fact that
> >    the primary limiting mechanism in the unified hierarchy is
> >    memory.high while memory.limit is either left unset or set to a very
> >    large value, moving memsw.limit knob to the unified hierarchy would
> >    effectively make it impossible to limit swap usage according to the
> >    user preference.
> > 
> >  - memsw.usage != memory.usage + swap.usage, because a page occupying
> >    both swap entry and a swap cache page is charged only once to memsw
> >    counter. As a result, it is possible to effectively eat up to
> >    memory.limit of memory pages *and* memsw.limit of swap entries, which
> >    looks unexpected.
> > 
> > That said, we should provide a different swap limiting mechanism for
> > cgroup2.
> > This patch adds mem_cgroup->swap counter, which charges the actual
> > number of swap entries used by a cgroup. It is only charged in the
> > unified hierarchy, while the legacy hierarchy memsw logic is left
> > intact.
> 
> I agree that the previous semantic was awkward. The problem I can see
> with this approach is that once the swap limit is reached the anon
> memory pressure might spill over to other and unrelated memcgs during
> the global memory pressure. I guess this is what Kame referred to as
> anon would become mlocked basically. This would be even more of an issue
> with resource delegation to sub-hierarchies because nobody will prevent
> setting the swap amount to a small value and use that as an anon memory
> protection.

Overcommitting untrusted workloads is already problematic because
reclaim is based on heuristics and references, and a malicious
workload can already interfere with it and create pressure on the
system or its neighboring groups. This patch doesn't make it better,
but it's not a new problem.

If you don't trust subhierarchies, don't give them more memory than
you can handle them taking. And then giving them swap is a resource
for them to use on top of that memory, not for you at the toplevel.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1291511

On Mon, Dec 14, 2015 at 04:30:37PM +0100, Michal Hocko wrote:
> On Thu 10-12-15 14:39:14, Vladimir Davydov wrote:
> > In the legacy hierarchy we charge memsw, which is dubious, because:
> > 
> >  - memsw.limit must be >= memory.limit, so it is impossible to limit
> >    swap usage less than memory usage. Taking into account the fact that
> >    the primary limiting mechanism in the unified hierarchy is
> >    memory.high while memory.limit is either left unset or set to a very
> >    large value, moving memsw.limit knob to the unified hierarchy would
> >    effectively make it impossible to limit swap usage according to the
> >    user preference.
> > 
> >  - memsw.usage != memory.usage + swap.usage, because a page occupying
> >    both swap entry and a swap cache page is charged only once to memsw
> >    counter. As a result, it is possible to effectively eat up to
> >    memory.limit of memory pages *and* memsw.limit of swap entries, which
> >    looks unexpected.
> > 
> > That said, we should provide a different swap limiting mechanism for
> > cgroup2.
> > This patch adds mem_cgroup->swap counter, which charges the actual
> > number of swap entries used by a cgroup. It is only charged in the
> > unified hierarchy, while the legacy hierarchy memsw logic is left
> > intact.
> 
> I agree that the previous semantic was awkward. The problem I can see
> with this approach is that once the swap limit is reached the anon
> memory pressure might spill over to other and unrelated memcgs during
> the global memory pressure. I guess this is what Kame referred to as
> anon would become mlocked basically. This would be even more of an issue
> with resource delegation to sub-hierarchies because nobody will prevent
> setting the swap amount to a small value and use that as an anon memory
> protection.

AFAICS such anon memory protection has a side-effect: real-life
workloads need page cache to run smoothly (at least for mapping
executables). Disabling swapping would switch pressure to page caches,
resulting in performance degradation. So, I don't think per memcg swap
limit can be abused to boost your workload on an overcommitted system.

If you mean malicious users, well, they already have plenty ways to eat
all available memory up to the hard limit by creating unreclaimable
kernel objects.

Anyway, if you don't trust a container you'd better set the hard memory
limit so that it can't hurt others no matter what it runs and how it
tweaks its sub-tree knobs.

...
> My question now is. Is the knob usable/useful even without additional
> heuristics? Do we want to protect swap space so rigidly that a swap
> limited memcg can cause bigger problems than without the swap limit
> globally?

Hmm, I don't see why problems might get bigger with per memcg swap limit
than w/o it. W/o swap limit, a memcg can eat all swap space on the host
and disable swapping for everyone, not just for itself alone.

Thanks,
Vladimir
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1291513

> Anyway, if you don't trust a container you'd better set the hard memory
> limit so that it can't hurt others no matter what it runs and how it
> tweaks its sub-tree knobs.

If you don't trust it put it in a VM. If it's got access to GEM graphics
ioctls/nodes or some other kernel interfaces then it can blow up the
kernel without trying hard unless its constrained within a VM. VMs can
be extremely light weight if you avoid KVM emulating an entire PC.

Alan
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1291813

On 2015/12/15 4:42, Vladimir Davydov wrote:
> On Mon, Dec 14, 2015 at 04:30:37PM +0100, Michal Hocko wrote:
>> On Thu 10-12-15 14:39:14, Vladimir Davydov wrote:
>>> In the legacy hierarchy we charge memsw, which is dubious, because:
>>>
>>>   - memsw.limit must be >= memory.limit, so it is impossible to limit
>>>     swap usage less than memory usage. Taking into account the fact that
>>>     the primary limiting mechanism in the unified hierarchy is
>>>     memory.high while memory.limit is either left unset or set to a very
>>>     large value, moving memsw.limit knob to the unified hierarchy would
>>>     effectively make it impossible to limit swap usage according to the
>>>     user preference.
>>>
>>>   - memsw.usage != memory.usage + swap.usage, because a page occupying
>>>     both swap entry and a swap cache page is charged only once to memsw
>>>     counter. As a result, it is possible to effectively eat up to
>>>     memory.limit of memory pages *and* memsw.limit of swap entries, which
>>>     looks unexpected.
>>>
>>> That said, we should provide a different swap limiting mechanism for
>>> cgroup2.
>>> This patch adds mem_cgroup->swap counter, which charges the actual
>>> number of swap entries used by a cgroup. It is only charged in the
>>> unified hierarchy, while the legacy hierarchy memsw logic is left
>>> intact.
>>
>> I agree that the previous semantic was awkward. The problem I can see
>> with this approach is that once the swap limit is reached the anon
>> memory pressure might spill over to other and unrelated memcgs during
>> the global memory pressure. I guess this is what Kame referred to as
>> anon would become mlocked basically. This would be even more of an issue
>> with resource delegation to sub-hierarchies because nobody will prevent
>> setting the swap amount to a small value and use that as an anon memory
>> protection.
>
> AFAICS such anon memory protection has a side-effect: real-life
> workloads need page cache to run smoothly (at least for mapping
> executables). Disabling swapping would switch pressure to page caches,
> resulting in performance degradation. So, I don't think per memcg swap
> limit can be abused to boost your workload on an overcommitted system.
>
> If you mean malicious users, well, they already have plenty ways to eat
> all available memory up to the hard limit by creating unreclaimable
> kernel objects.
>
"protect anon" user's malicious degree is far lower than such cracker like users.

> Anyway, if you don't trust a container you'd better set the hard memory
> limit so that it can't hurt others no matter what it runs and how it
> tweaks its sub-tree knobs.
>

Limiting swap can easily cause "OOM-Killer even while there are available swap"
with easy mistake. Can't you add "swap excess" switch to sysctl to allow global
memory reclaim can ignore swap limitation ?

Regards,
-Kame








--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1292044

On Tue, Dec 15, 2015 at 12:22:41PM +0900, Kamezawa Hiroyuki wrote:
> On 2015/12/15 4:42, Vladimir Davydov wrote:
> >On Mon, Dec 14, 2015 at 04:30:37PM +0100, Michal Hocko wrote:
> >>On Thu 10-12-15 14:39:14, Vladimir Davydov wrote:
> >>>In the legacy hierarchy we charge memsw, which is dubious, because:
> >>>
> >>>  - memsw.limit must be >= memory.limit, so it is impossible to limit
> >>>    swap usage less than memory usage. Taking into account the fact that
> >>>    the primary limiting mechanism in the unified hierarchy is
> >>>    memory.high while memory.limit is either left unset or set to a very
> >>>    large value, moving memsw.limit knob to the unified hierarchy would
> >>>    effectively make it impossible to limit swap usage according to the
> >>>    user preference.
> >>>
> >>>  - memsw.usage != memory.usage + swap.usage, because a page occupying
> >>>    both swap entry and a swap cache page is charged only once to memsw
> >>>    counter. As a result, it is possible to effectively eat up to
> >>>    memory.limit of memory pages *and* memsw.limit of swap entries, which
> >>>    looks unexpected.
> >>>
> >>>That said, we should provide a different swap limiting mechanism for
> >>>cgroup2.
> >>>This patch adds mem_cgroup->swap counter, which charges the actual
> >>>number of swap entries used by a cgroup. It is only charged in the
> >>>unified hierarchy, while the legacy hierarchy memsw logic is left
> >>>intact.
> >>
> >>I agree that the previous semantic was awkward. The problem I can see
> >>with this approach is that once the swap limit is reached the anon
> >>memory pressure might spill over to other and unrelated memcgs during
> >>the global memory pressure. I guess this is what Kame referred to as
> >>anon would become mlocked basically. This would be even more of an issue
> >>with resource delegation to sub-hierarchies because nobody will prevent
> >>setting the swap amount to a small value and use that as an anon memory
> >>protection.
> >
> >AFAICS such anon memory protection has a side-effect: real-life
> >workloads need page cache to run smoothly (at least for mapping
> >executables). Disabling swapping would switch pressure to page caches,
> >resulting in performance degradation. So, I don't think per memcg swap
> >limit can be abused to boost your workload on an overcommitted system.
> >
> >If you mean malicious users, well, they already have plenty ways to eat
> >all available memory up to the hard limit by creating unreclaimable
> >kernel objects.
> >
> "protect anon" user's malicious degree is far lower than such cracker like users.

What do you mean by "malicious degree"? What is such a user trying to
achieve? Killing the system? Well, there are much more effective ways to
do so. Or does it want to exploit a system specific feature to get
benefit for itself? If so, it will hardly win by mlocking all anonymous
memory, because this will result in higher pressure exerted upon its
page cache and dcache, which normal workloads just can't get along
without.

> 
> >Anyway, if you don't trust a container you'd better set the hard memory
> >limit so that it can't hurt others no matter what it runs and how it
> >tweaks its sub-tree knobs.
> >
> 
> Limiting swap can easily cause "OOM-Killer even while there are
> available swap" with easy mistake.

What do you mean by "easy mistake"? Misconfiguration? If so, it's a lame
excuse IMO. Admin should take system configuration seriously. If the
host is not overcommitted, it's trivial. Otherwise, there's always a
chance that things will go south, so it's not going to be easy. It's up
to admin to analyze risks and set limits accordingly. Exporting knobs
with clear meaning is the best we can do here. swap.max is one such knob
It defines maximal usage of swap resource. Allowing to breach it just
does not add up.

> Can't you add "swap excess" switch to sysctl to allow global memory
> reclaim can ignore swap limitation ?

I'd be opposed to it, because this would obscure the user API. OTOH, a
kind of swap soft limit (swap.high?) might be considered. I'm not sure
if it's really necessary though, because all arguments for it do not
look convincing to me for now. So, personally, I would refrain from
implementing it until it is really called for by users of cgroup v2.

Thanks,
Vladimir
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1292710

On 2015/12/15 20:02, Vladimir Davydov wrote:
> On Tue, Dec 15, 2015 at 12:22:41PM +0900, Kamezawa Hiroyuki wrote:
>> On 2015/12/15 4:42, Vladimir Davydov wrote:
>>> On Mon, Dec 14, 2015 at 04:30:37PM +0100, Michal Hocko wrote:
>>>> On Thu 10-12-15 14:39:14, Vladimir Davydov wrote:
>>>>> In the legacy hierarchy we charge memsw, which is dubious, because:
>>>>>
>>>>>   - memsw.limit must be >= memory.limit, so it is impossible to limit
>>>>>     swap usage less than memory usage. Taking into account the fact that
>>>>>     the primary limiting mechanism in the unified hierarchy is
>>>>>     memory.high while memory.limit is either left unset or set to a very
>>>>>     large value, moving memsw.limit knob to the unified hierarchy would
>>>>>     effectively make it impossible to limit swap usage according to the
>>>>>     user preference.
>>>>>
>>>>>   - memsw.usage != memory.usage + swap.usage, because a page occupying
>>>>>     both swap entry and a swap cache page is charged only once to memsw
>>>>>     counter. As a result, it is possible to effectively eat up to
>>>>>     memory.limit of memory pages *and* memsw.limit of swap entries, which
>>>>>     looks unexpected.
>>>>>
>>>>> That said, we should provide a different swap limiting mechanism for
>>>>> cgroup2.
>>>>> This patch adds mem_cgroup->swap counter, which charges the actual
>>>>> number of swap entries used by a cgroup. It is only charged in the
>>>>> unified hierarchy, while the legacy hierarchy memsw logic is left
>>>>> intact.
>>>>
>>>> I agree that the previous semantic was awkward. The problem I can see
>>>> with this approach is that once the swap limit is reached the anon
>>>> memory pressure might spill over to other and unrelated memcgs during
>>>> the global memory pressure. I guess this is what Kame referred to as
>>>> anon would become mlocked basically. This would be even more of an issue
>>>> with resource delegation to sub-hierarchies because nobody will prevent
>>>> setting the swap amount to a small value and use that as an anon memory
>>>> protection.
>>>
>>> AFAICS such anon memory protection has a side-effect: real-life
>>> workloads need page cache to run smoothly (at least for mapping
>>> executables). Disabling swapping would switch pressure to page caches,
>>> resulting in performance degradation. So, I don't think per memcg swap
>>> limit can be abused to boost your workload on an overcommitted system.
>>>
>>> If you mean malicious users, well, they already have plenty ways to eat
>>> all available memory up to the hard limit by creating unreclaimable
>>> kernel objects.
>>>
>> "protect anon" user's malicious degree is far lower than such cracker like users.
>
> What do you mean by "malicious degree"? What is such a user trying to
> achieve? Killing the system? Well, there are much more effective ways to
> do so. Or does it want to exploit a system specific feature to get
> benefit for itself? If so, it will hardly win by mlocking all anonymous
> memory, because this will result in higher pressure exerted upon its
> page cache and dcache, which normal workloads just can't get along
> without.
>

I wanted to say almost all application developers want to set swap.limit=0 if allowed.
So, it's a usual people who can kill the system if swap imbalance is allowed.
  
>>
>>> Anyway, if you don't trust a container you'd better set the hard memory
>>> limit so that it can't hurt others no matter what it runs and how it
>>> tweaks its sub-tree knobs.
>>>
>>
>> Limiting swap can easily cause "OOM-Killer even while there are
>> available swap" with easy mistake.
>
> What do you mean by "easy mistake"? Misconfiguration? If so, it's a lame
> excuse IMO. Admin should take system configuration seriously. If the
> host is not overcommitted, it's trivial. Otherwise, there's always a
> chance that things will go south, so it's not going to be easy. It's up
> to admin to analyze risks and set limits accordingly. Exporting knobs
> with clear meaning is the best we can do here. swap.max is one such knob
> It defines maximal usage of swap resource. Allowing to breach it just
> does not add up.
>
>> Can't you add "swap excess" switch to sysctl to allow global memory
>> reclaim can ignore swap limitation ?
>
> I'd be opposed to it, because this would obscure the user API. OTOH, a
> kind of swap soft limit (swap.high?) might be considered. I'm not sure
> if it's really necessary though, because all arguments for it do not
> look convincing to me for now. So, personally, I would refrain from
> implementing it until it is really called for by users of cgroup v2.
>

Considering my customers, running OOM-Killer while there are free swap space is
system's error rather than their misconfiguration.

BTW, mlock() requires CAP_IPC_LOCK.
please set default unlimited and check capability at setting swap limit, at least.

Thanks,
-Kame


> Thanks,
> Vladimir
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1292237

On Tue, Dec 15, 2015 at 12:22:41PM +0900, Kamezawa Hiroyuki wrote:
> On 2015/12/15 4:42, Vladimir Davydov wrote:
> >Anyway, if you don't trust a container you'd better set the hard memory
> >limit so that it can't hurt others no matter what it runs and how it
> >tweaks its sub-tree knobs.
> 
> Limiting swap can easily cause "OOM-Killer even while there are available swap"
> with easy mistake. Can't you add "swap excess" switch to sysctl to allow global
> memory reclaim can ignore swap limitation ?

That never worked with a combined memory+swap limit, either. How could
it? The parent might swap you out under pressure, but simply touching
a few of your anon pages causes them to get swapped back in, thrashing
with whatever the parent was trying to do. Your ability to swap it out
is simply no protection against a group touching its pages.

Allowing the parent to exceed swap with separate counters makes even
less sense, because every page swapped out frees up a page of memory
that the child can reuse. For every swap page that exceeds the limit,
the child gets a free memory page! The child doesn't even have to
cause swapin, it can just steal whatever the parent tried to free up,
and meanwhile its combined memory & swap footprint explodes.

The answer is and always should have been: don't overcommit untrusted
cgroups. Think of swap as a resource you distribute, not as breathing
room for the parents to rely on. Because it can't and could never.

And the new separate swap counter makes this explicit.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1292719

On 2015/12/15 23:50, Johannes Weiner wrote:
> On Tue, Dec 15, 2015 at 12:22:41PM +0900, Kamezawa Hiroyuki wrote:
>> On 2015/12/15 4:42, Vladimir Davydov wrote:
>>> Anyway, if you don't trust a container you'd better set the hard memory
>>> limit so that it can't hurt others no matter what it runs and how it
>>> tweaks its sub-tree knobs.
>>
>> Limiting swap can easily cause "OOM-Killer even while there are available swap"
>> with easy mistake. Can't you add "swap excess" switch to sysctl to allow global
>> memory reclaim can ignore swap limitation ?
>
> That never worked with a combined memory+swap limit, either. How could
> it? The parent might swap you out under pressure, but simply touching
> a few of your anon pages causes them to get swapped back in, thrashing
> with whatever the parent was trying to do. Your ability to swap it out
> is simply no protection against a group touching its pages.
>
> Allowing the parent to exceed swap with separate counters makes even
> less sense, because every page swapped out frees up a page of memory
> that the child can reuse. For every swap page that exceeds the limit,
> the child gets a free memory page! The child doesn't even have to
> cause swapin, it can just steal whatever the parent tried to free up,
> and meanwhile its combined memory & swap footprint explodes.
>
Sure.

> The answer is and always should have been: don't overcommit untrusted
> cgroups. Think of swap as a resource you distribute, not as breathing
> room for the parents to rely on. Because it can't and could never.
>
ok, don't overcommmit.

> And the new separate swap counter makes this explicit.
>
Hmm, my requests are
  - set the same capabilities as mlock() to set swap.limit=0
  - swap-full notification via vmpressure or something mechanism.
  - OOM-Killer's available memory calculation may be corrupted, please check.
  - force swap-in at reducing swap.limit

Thanks,
-Kame


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1292899

On Wed, Dec 16, 2015 at 12:18:30PM +0900, Kamezawa Hiroyuki wrote:
> Hmm, my requests are
>  - set the same capabilities as mlock() to set swap.limit=0

Setting swap.max is already privileged operation.

>  - swap-full notification via vmpressure or something mechanism.

Why?

>  - OOM-Killer's available memory calculation may be corrupted, please check.

Vladimir updated mem_cgroup_get_limit().

>  - force swap-in at reducing swap.limit

Why?
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1293563

On 2015/12/16 20:09, Johannes Weiner wrote:
> On Wed, Dec 16, 2015 at 12:18:30PM +0900, Kamezawa Hiroyuki wrote:
>> Hmm, my requests are
>>   - set the same capabilities as mlock() to set swap.limit=0
>
> Setting swap.max is already privileged operation.
>
Sure.

>>   - swap-full notification via vmpressure or something mechanism.
>
> Why?
>

I think it's a sign of unhealthy condition, starting file cache drop rate to rise.
But I forgot that there are resource threshold notifier already. Does the notifier work
for swap.usage ?

>>   - OOM-Killer's available memory calculation may be corrupted, please check.
>
> Vladimir updated mem_cgroup_get_limit().
>
I'll check it.

>>   - force swap-in at reducing swap.limit
>
> Why?
>
If full, swap.limit cannot be reduced even if there are available memory in a cgroup.
Another cgroup cannot make use of the swap resource while it's occupied by other cgroup.
The job scheduler should have a chance to fix the situation.

Thanks,
-Kame

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1293595

On Thu, Dec 17, 2015 at 11:46:27AM +0900, Kamezawa Hiroyuki wrote:
> On 2015/12/16 20:09, Johannes Weiner wrote:
> >On Wed, Dec 16, 2015 at 12:18:30PM +0900, Kamezawa Hiroyuki wrote:
> >>  - swap-full notification via vmpressure or something mechanism.
> >
> >Why?
> >
> 
> I think it's a sign of unhealthy condition, starting file cache drop rate to rise.
> But I forgot that there are resource threshold notifier already. Does the notifier work
> for swap.usage ?

That will be reflected in vmpressure or other distress mechanisms. I'm
not convinced "ran out of swap space" needs special casing in any way.

> >>  - force swap-in at reducing swap.limit
> >
> >Why?
> >
> If full, swap.limit cannot be reduced even if there are available memory in a cgroup.
> Another cgroup cannot make use of the swap resource while it's occupied by other cgroup.
> The job scheduler should have a chance to fix the situation.

I don't see why swap space allowance would need to be as dynamically
adjustable as the memory allowance. There is usually no need to be as
tight with swap space as with memory, and the performance penalty of
swapping, even with flash drives, is high enough that swap space acts
as an overflow vessel rather than be part of the regularly backing of
the anonymous/shmem working set. It really is NOT obvious that swap
space would need to be adjusted on the fly, and that it's important
that reducing the limit will be reflected in consumption right away.

We shouldn't be adding hundreds of lines of likely terrible heuristics
code* on speculation that somebody MIGHT find this useful in real life.
We should wait until we are presented with a real usecase that applies
to a whole class of users, and then see what the true requirements are.

* If a group has 200M swapped out and the swap limit is reduced by 10M
below the current consumption, which pages would you swap in? There is
no LRU list for swap space.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1293606

On 2015/12/17 12:32, Johannes Weiner wrote:
> On Thu, Dec 17, 2015 at 11:46:27AM +0900, Kamezawa Hiroyuki wrote:
>> On 2015/12/16 20:09, Johannes Weiner wrote:
>>> On Wed, Dec 16, 2015 at 12:18:30PM +0900, Kamezawa Hiroyuki wrote:
>>>>   - swap-full notification via vmpressure or something mechanism.
>>>
>>> Why?
>>>
>>
>> I think it's a sign of unhealthy condition, starting file cache drop rate to rise.
>> But I forgot that there are resource threshold notifier already. Does the notifier work
>> for swap.usage ?
>
> That will be reflected in vmpressure or other distress mechanisms. I'm
> not convinced "ran out of swap space" needs special casing in any way.
>
Most users checks swap space shortage as "system alarm" in enterprise systems.
At least, our customers checks swap-full.

>>>>   - force swap-in at reducing swap.limit
>>>
>>> Why?
>>>
>> If full, swap.limit cannot be reduced even if there are available memory in a cgroup.
>> Another cgroup cannot make use of the swap resource while it's occupied by other cgroup.
>> The job scheduler should have a chance to fix the situation.
>
> I don't see why swap space allowance would need to be as dynamically
> adjustable as the memory allowance. There is usually no need to be as
> tight with swap space as with memory, and the performance penalty of
> swapping, even with flash drives, is high enough that swap space acts
> as an overflow vessel rather than be part of the regularly backing of
> the anonymous/shmem working set. It really is NOT obvious that swap
> space would need to be adjusted on the fly, and that it's important
> that reducing the limit will be reflected in consumption right away.
>

With my OS support experience, some customers consider swap-space as a resource.


> We shouldn't be adding hundreds of lines of likely terrible heuristics
> code* on speculation that somebody MIGHT find this useful in real life.
> We should wait until we are presented with a real usecase that applies
> to a whole class of users, and then see what the true requirements are.
>
ok, we should wait.  I'm just guessing (japanese) HPC people will want the
feature for their job control. I hear many programs relies on swap.

> * If a group has 200M swapped out and the swap limit is reduced by 10M
> below the current consumption, which pages would you swap in? There is
> no LRU list for swap space.
>
If a rotation can happen when a swap-in-by-real-pagefault, random swap-in
at reducing swap.limit will work enough.

Thanks,
-Kame



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1292383

On Mon 14-12-15 22:42:58, Vladimir Davydov wrote:
> On Mon, Dec 14, 2015 at 04:30:37PM +0100, Michal Hocko wrote:
> > On Thu 10-12-15 14:39:14, Vladimir Davydov wrote:
> > > In the legacy hierarchy we charge memsw, which is dubious, because:
> > > 
> > >  - memsw.limit must be >= memory.limit, so it is impossible to limit
> > >    swap usage less than memory usage. Taking into account the fact that
> > >    the primary limiting mechanism in the unified hierarchy is
> > >    memory.high while memory.limit is either left unset or set to a very
> > >    large value, moving memsw.limit knob to the unified hierarchy would
> > >    effectively make it impossible to limit swap usage according to the
> > >    user preference.
> > > 
> > >  - memsw.usage != memory.usage + swap.usage, because a page occupying
> > >    both swap entry and a swap cache page is charged only once to memsw
> > >    counter. As a result, it is possible to effectively eat up to
> > >    memory.limit of memory pages *and* memsw.limit of swap entries, which
> > >    looks unexpected.
> > > 
> > > That said, we should provide a different swap limiting mechanism for
> > > cgroup2.
> > > This patch adds mem_cgroup->swap counter, which charges the actual
> > > number of swap entries used by a cgroup. It is only charged in the
> > > unified hierarchy, while the legacy hierarchy memsw logic is left
> > > intact.
> > 
> > I agree that the previous semantic was awkward. The problem I can see
> > with this approach is that once the swap limit is reached the anon
> > memory pressure might spill over to other and unrelated memcgs during
> > the global memory pressure. I guess this is what Kame referred to as
> > anon would become mlocked basically. This would be even more of an issue
> > with resource delegation to sub-hierarchies because nobody will prevent
> > setting the swap amount to a small value and use that as an anon memory
> > protection.
> 
> AFAICS such anon memory protection has a side-effect: real-life
> workloads need page cache to run smoothly (at least for mapping
> executables). Disabling swapping would switch pressure to page caches,
> resulting in performance degradation. So, I don't think per memcg swap
> limit can be abused to boost your workload on an overcommitted system.

Well, you can trash on the page cache which could slow down the workload
but the executable pages get an additional protection so this might be
not sufficient and still trigger a massive disruption on the global level.

> If you mean malicious users, well, they already have plenty ways to eat
> all available memory up to the hard limit by creating unreclaimable
> kernel objects.
> 
> Anyway, if you don't trust a container you'd better set the hard memory
> limit so that it can't hurt others no matter what it runs and how it
> tweaks its sub-tree knobs.

I completely agree that malicious/untrusted users absolutely have to
be capped by the hard limit. Then the separate swap limit would work
for sure. But I am less convinced about usefulness of the rigid (to
the global memory pressure) swap limit without the hard limit. All the
memory that could have been swapped out will make a memory pressure to
the rest of the system without being punished for it too much. Memcg
is allowed to grow over the high limit (in the current implementation)
without any way to shrink back in other words.

My understanding was that the primary use case for the swap limit is to
handle potential (not only malicious but also unexpectedly misbehaving
application) anon memory consumption runaways more gracefully without
the massive disruption on the global level. I simply didn't see swap
space partitioning as important enough because an alternative to swap
usage is to consume primary memory which is a more precious resource
IMO. Swap storage is really cheap and runtime expandable resource which
is not the case for the primary memory in general. Maybe there are other
use cases I am not aware of, though. Do you want to guarantee the swap
availability?

Just to make it clear. I am not against the new way of the swap
accounting. It is much more clear then the previous one. I am just
worried it allows for an easy misconfiguration and we do not have any
measures to help the global system healthiness. I am OK with the patch
if we document the risk for now. I still think we will end up doing some
heuristic to throttle for a large unreclaimable high limit excess in the
future but I agree this shouldn't be the prerequisite.

> ...
> > My question now is. Is the knob usable/useful even without additional
> > heuristics? Do we want to protect swap space so rigidly that a swap
> > limited memcg can cause bigger problems than without the swap limit
> > globally?
> 
> Hmm, I don't see why problems might get bigger with per memcg swap limit
> than w/o it.

because the reclaim fairness between different memcg hierarchies will be
severely affected.

> W/o swap limit, a memcg can eat all swap space on the host
> and disable swapping for everyone, not just for itself alone.

this is true of course but this is not very much different from pushing
everybody else to the swap while eating the unreclaimable anonymous
memory and eventually hit the OOM killer.

-- 
Michal Hocko
SUSE Labs
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1292519

On Tue, Dec 15, 2015 at 06:21:28PM +0100, Michal Hocko wrote:
> > AFAICS such anon memory protection has a side-effect: real-life
> > workloads need page cache to run smoothly (at least for mapping
> > executables). Disabling swapping would switch pressure to page caches,
> > resulting in performance degradation. So, I don't think per memcg swap
> > limit can be abused to boost your workload on an overcommitted system.
> 
> Well, you can trash on the page cache which could slow down the workload
> but the executable pages get an additional protection so this might be
> not sufficient and still trigger a massive disruption on the global level.

No, this is a real consequence. If you fill your available memory with
mostly unreclaimable memory and your executables start thrashing you
might not make forward progress for hours. We don't have a swap token
for page cache.

> Just to make it clear. I am not against the new way of the swap
> accounting. It is much more clear then the previous one. I am just
> worried it allows for an easy misconfiguration and we do not have any
> measures to help the global system healthiness. I am OK with the patch
> if we document the risk for now. I still think we will end up doing some
> heuristic to throttle for a large unreclaimable high limit excess in the
> future but I agree this shouldn't be the prerequisite.

It's unclear to me how the previous memory+swap counters did anything
tangible for global system health with malicious/buggy workloads. If
anything, the previous model seems to encourage blatant overcommit of
workloads on the flawed assumption that global pressure could always
claw back memory, including anonymous pages of untrusted workloads,
which does not actually work in practice. So I'm not sure what new
risk you are referring to here.

As far as the high limit goes, its job is to contain cache growth and
throttle applications during somewhat higher-than-expected consumption
peaks; not to contain "large unreclaimable high limit excess" from
buggy or malicious applications, that's what the hard limit is for.

All in all, it seems to me we should leave this discussion to actual
problems arising in the real world. There is a lot of unfocussed
speculation in this thread about things that might go wrong, without
much thought put into whether these scenarios are even meaningful or
real or whether they are new problems that come with the swap limit.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1292748

On 2015/12/16 2:21, Michal Hocko wrote:
  
> I completely agree that malicious/untrusted users absolutely have to
> be capped by the hard limit. Then the separate swap limit would work
> for sure. But I am less convinced about usefulness of the rigid (to
> the global memory pressure) swap limit without the hard limit. All the
> memory that could have been swapped out will make a memory pressure to
> the rest of the system without being punished for it too much. Memcg
> is allowed to grow over the high limit (in the current implementation)
> without any way to shrink back in other words.
>
> My understanding was that the primary use case for the swap limit is to
> handle potential (not only malicious but also unexpectedly misbehaving
> application) anon memory consumption runaways more gracefully without
> the massive disruption on the global level. I simply didn't see swap
> space partitioning as important enough because an alternative to swap
> usage is to consume primary memory which is a more precious resource
> IMO. Swap storage is really cheap and runtime expandable resource which
> is not the case for the primary memory in general. Maybe there are other
> use cases I am not aware of, though. Do you want to guarantee the swap
> availability?
>

At the first implementation, NEC guy explained their use case in HPC area.
At that time, there was no swap support.

Considering 2 workloads partitioned into group A, B. total swap was 100GB.
   A: memory.limit = 40G
   B: memory.limit = 40G

Job scheduler runs applications in A and B in turn. Apps in A stops while Apps in B running.

If App-A requires 120GB of anonymous memory, it uses 80GB of swap. So, App-B can use only
20GB of swap. This can cause trouble if App-B needs 100GB of anonymous memory.
They need some knob to control amount of swap per cgroup.

The point is, at least for their customer, the swap is "resource", which should be under
control. With their use case, memory usage and swap usage has the same meaning. So,
mem+swap limit doesn't cause trouble.

Thanks,
-Kame




--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1291811

On 2015/12/15 0:30, Michal Hocko wrote:
> On Thu 10-12-15 14:39:14, Vladimir Davydov wrote:
>> In the legacy hierarchy we charge memsw, which is dubious, because:
>>
>>   - memsw.limit must be >= memory.limit, so it is impossible to limit
>>     swap usage less than memory usage. Taking into account the fact that
>>     the primary limiting mechanism in the unified hierarchy is
>>     memory.high while memory.limit is either left unset or set to a very
>>     large value, moving memsw.limit knob to the unified hierarchy would
>>     effectively make it impossible to limit swap usage according to the
>>     user preference.
>>
>>   - memsw.usage != memory.usage + swap.usage, because a page occupying
>>     both swap entry and a swap cache page is charged only once to memsw
>>     counter. As a result, it is possible to effectively eat up to
>>     memory.limit of memory pages *and* memsw.limit of swap entries, which
>>     looks unexpected.
>>
>> That said, we should provide a different swap limiting mechanism for
>> cgroup2.
>> This patch adds mem_cgroup->swap counter, which charges the actual
>> number of swap entries used by a cgroup. It is only charged in the
>> unified hierarchy, while the legacy hierarchy memsw logic is left
>> intact.
>
> I agree that the previous semantic was awkward. The problem I can see
> with this approach is that once the swap limit is reached the anon
> memory pressure might spill over to other and unrelated memcgs during
> the global memory pressure. I guess this is what Kame referred to as
> anon would become mlocked basically. This would be even more of an issue
> with resource delegation to sub-hierarchies because nobody will prevent
> setting the swap amount to a small value and use that as an anon memory
> protection.
>
> I guess this was the reason why this approach hasn't been chosen before

Yes. At that age, "never break global VM" was the policy. And "mlock" can be
used for attacking system.

> but I think we can come up with a way to stop the run away consumption
> even when the swap is accounted separately. All of them are quite nasty
> but let me try.
>
> We could allow charges to fail even for the high limit if the excess is
> way above the amount of reclaimable memory in the given memcg/hierarchy.
> A runaway load would be stopped before it can cause a considerable
> damage outside of its hierarchy this way even when the swap limit
> is configured small.
> Now that goes against the high limit semantic which should only throttle
> the consumer and shouldn't cause any functional failures but maybe this
> is acceptable for the overall system stability. An alternative would
> be to throttle in the high limit reclaim context proportionally to
> the excess. This is normally done by the reclaim itself but with no
> reclaimable memory this wouldn't work that way.
>
This seems hard to use for users who want to control resource precisely
even if stability is good.

> Another option would be to ignore the swap limit during the global
> reclaim. This wouldn't stop the runaway loads but they would at least
> see their fair share of the reclaim. The swap excess could be then used
> as a "handicap" for a more aggressive throttling during high limit reclaim
> or to trigger hard limit sooner.
>
This seems to work. But users need to understand swap-limit can be exceeded.

> Or we could teach the global OOM killer to select abusive anon memory
> users with restricted swap. That would require to iterate through all
> memcgs and checks whether their anon consumption is in a large excess to
> their swap limit and fallback to the memcg OOM victim selection if that
> is the case. This adds more complexity to the OOM killer path so I am
> not sure this is generally acceptable, though.
>

I think this is not acceptable.

> My question now is. Is the knob usable/useful even without additional
> heuristics? Do we want to protect swap space so rigidly that a swap
> limited memcg can cause bigger problems than without the swap limit
> globally?
>

swap requires some limit. If not, an application can eat up all swap
and it will not be never freed until the application access it or
swapoff runs.

Thanks,
-Kame

>> The swap usage can be monitored using new memory.swap.current file and
>> limited using memory.swap.max.
>>
>> Signed-off-by: Vladimir Davydov <vdavydov@virtuozzo.com>
>> ---
>>   include/linux/memcontrol.h |   1 +
>>   include/linux/swap.h       |   5 ++
>>   mm/memcontrol.c            | 123 +++++++++++++++++++++++++++++++++++++++++----
>>   mm/shmem.c                 |   4 ++
>>   mm/swap_state.c            |   5 ++
>>   5 files changed, 129 insertions(+), 9 deletions(-)
>
> [...]
>


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1291933

On Tue, Dec 15, 2015 at 12:12:40PM +0900, Kamezawa Hiroyuki wrote:
> On 2015/12/15 0:30, Michal Hocko wrote:
> >On Thu 10-12-15 14:39:14, Vladimir Davydov wrote:
> >>In the legacy hierarchy we charge memsw, which is dubious, because:
> >>
> >>  - memsw.limit must be >= memory.limit, so it is impossible to limit
> >>    swap usage less than memory usage. Taking into account the fact that
> >>    the primary limiting mechanism in the unified hierarchy is
> >>    memory.high while memory.limit is either left unset or set to a very
> >>    large value, moving memsw.limit knob to the unified hierarchy would
> >>    effectively make it impossible to limit swap usage according to the
> >>    user preference.
> >>
> >>  - memsw.usage != memory.usage + swap.usage, because a page occupying
> >>    both swap entry and a swap cache page is charged only once to memsw
> >>    counter. As a result, it is possible to effectively eat up to
> >>    memory.limit of memory pages *and* memsw.limit of swap entries, which
> >>    looks unexpected.
> >>
> >>That said, we should provide a different swap limiting mechanism for
> >>cgroup2.
> >>This patch adds mem_cgroup->swap counter, which charges the actual
> >>number of swap entries used by a cgroup. It is only charged in the
> >>unified hierarchy, while the legacy hierarchy memsw logic is left
> >>intact.
> >
> >I agree that the previous semantic was awkward. The problem I can see
> >with this approach is that once the swap limit is reached the anon
> >memory pressure might spill over to other and unrelated memcgs during
> >the global memory pressure. I guess this is what Kame referred to as
> >anon would become mlocked basically. This would be even more of an issue
> >with resource delegation to sub-hierarchies because nobody will prevent
> >setting the swap amount to a small value and use that as an anon memory
> >protection.
> >
> >I guess this was the reason why this approach hasn't been chosen before
> 
> Yes. At that age, "never break global VM" was the policy. And "mlock" can be
> used for attacking system.

If we are talking about "attacking system" from inside a container,
there are much easier and disruptive ways, e.g. running a fork-bomb or
creating pipes - such memory can't be reclaimed and global OOM killer
won't help.

Thanks,
Vladimir
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1291976

On 2015/12/15 17:30, Vladimir Davydov wrote:
> On Tue, Dec 15, 2015 at 12:12:40PM +0900, Kamezawa Hiroyuki wrote:
>> On 2015/12/15 0:30, Michal Hocko wrote:
>>> On Thu 10-12-15 14:39:14, Vladimir Davydov wrote:
>>>> In the legacy hierarchy we charge memsw, which is dubious, because:
>>>>
>>>>   - memsw.limit must be >= memory.limit, so it is impossible to limit
>>>>     swap usage less than memory usage. Taking into account the fact that
>>>>     the primary limiting mechanism in the unified hierarchy is
>>>>     memory.high while memory.limit is either left unset or set to a very
>>>>     large value, moving memsw.limit knob to the unified hierarchy would
>>>>     effectively make it impossible to limit swap usage according to the
>>>>     user preference.
>>>>
>>>>   - memsw.usage != memory.usage + swap.usage, because a page occupying
>>>>     both swap entry and a swap cache page is charged only once to memsw
>>>>     counter. As a result, it is possible to effectively eat up to
>>>>     memory.limit of memory pages *and* memsw.limit of swap entries, which
>>>>     looks unexpected.
>>>>
>>>> That said, we should provide a different swap limiting mechanism for
>>>> cgroup2.
>>>> This patch adds mem_cgroup->swap counter, which charges the actual
>>>> number of swap entries used by a cgroup. It is only charged in the
>>>> unified hierarchy, while the legacy hierarchy memsw logic is left
>>>> intact.
>>>
>>> I agree that the previous semantic was awkward. The problem I can see
>>> with this approach is that once the swap limit is reached the anon
>>> memory pressure might spill over to other and unrelated memcgs during
>>> the global memory pressure. I guess this is what Kame referred to as
>>> anon would become mlocked basically. This would be even more of an issue
>>> with resource delegation to sub-hierarchies because nobody will prevent
>>> setting the swap amount to a small value and use that as an anon memory
>>> protection.
>>>
>>> I guess this was the reason why this approach hasn't been chosen before
>>
>> Yes. At that age, "never break global VM" was the policy. And "mlock" can be
>> used for attacking system.
>
> If we are talking about "attacking system" from inside a container,
> there are much easier and disruptive ways, e.g. running a fork-bomb or
> creating pipes - such memory can't be reclaimed and global OOM killer
> won't help.

You're right. We just wanted to avoid affecting global memory reclaim by
each cgroup settings.

Thanks,
-Kame



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web