[PATCH 0/8] mm: memcontrol: account socket memory in unified hierarchy

From	Johannes Weiner <hannes@cmpxchg.org>
Newsgroups	linux.kernel
Subject	[PATCH 0/8] mm: memcontrol: account socket memory in unified hierarchy
Date	2015-10-22 06:30 +0200
Message-ID	<qmfFL-87d-1@gated-at.bofh.it> (permalink)
Organization	linux.* mail to news gateway

Show all headers | View raw

Hi,

this series adds socket buffer memory tracking and accounting to the
unified hierarchy memory cgroup controller.

[ Networking people, at this time please check the diffstat below to
  avoid going into convulsions. ]

Socket buffer memory can make up a significant share of a workload's
memory footprint, and so it needs to be accounted and tracked out of
the box, along with other types of memory that can be directly linked
to userspace activity, in order to provide useful resource isolation.

Historically, socket buffers were accounted in a separate counter,
without any pressure equalization between anonymous memory, page
cache, and the socket buffers. When the socket buffer pool was
exhausted, buffer allocations would fail hard and cause network
performance to tank, regardless of whether there was still memory
available to the group or not. Likewise, struggling anonymous or cache
workingsets could not dip into an idle socket memory pool. Because of
this, the feature was not usable for many real life applications.

To not repeat this mistake, the new memory controller will account all
types of memory pages it is tracking on behalf of a cgroup in a single
pool. And upon pressure, the VM reclaims and shrinks whatever memory
in that pool is within its reach.

These patches add accounting for memory consumed by sockets associated
with a cgroup to the existing pool of anonymous pages and page cache.

Patch #3 reworks the existing memcg socket infrastructure. It has many
provisions for future plans that won't materialize, and much of this
simply evaporates. The networking people should be happy about this.

Patch #5 adds accounting and tracking of socket memory to the unified
hierarchy memory controller, as described above. It uses the existing
per-cpu charge caches and triggers high limit reclaim asynchroneously.

Patch #8 uses the vmpressure extension to equalize pressure between
the pages tracked natively by the VM and socket buffer pages. As the
pool is shared, it makes sense that while natively tracked pages are
under duress the network transmit windows are also not increased.

As per above, this is an essential part of the new memory controller's
core functionality. With the unified hierarchy nearing release, please
consider this for 4.4.

 include/linux/memcontrol.h   |  90 +++++++++-------
 include/linux/page_counter.h |   6 +-
 include/net/sock.h           | 139 ++----------------------
 include/net/tcp.h            |   5 +-
 include/net/tcp_memcontrol.h |   7 --
 mm/backing-dev.c             |   2 +-
 mm/hugetlb_cgroup.c          |   3 +-
 mm/memcontrol.c              | 235 ++++++++++++++++++++++++++---------------
 mm/page_counter.c            |  14 +--
 mm/vmpressure.c              |  29 ++++-
 mm/vmscan.c                  |  41 +++----
 net/core/sock.c              |  78 ++++----------
 net/ipv4/sysctl_net_ipv4.c   |   1 -
 net/ipv4/tcp.c               |   3 +-
 net/ipv4/tcp_ipv4.c          |   9 +-
 net/ipv4/tcp_memcontrol.c    | 147 ++++----------------------
 net/ipv4/tcp_output.c        |   6 +-
 net/ipv6/tcp_ipv6.c          |   3 -
 18 files changed, 319 insertions(+), 499 deletions(-)

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Back to linux.kernel | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

[PATCH 0/8] mm: memcontrol: account socket memory in unified hierarchy Johannes Weiner <hannes@cmpxchg.org> - 2015-10-22 06:30 +0200
  [PATCH 1/8] mm: page_counter: let page_counter_try_charge() return bool Johannes Weiner <hannes@cmpxchg.org> - 2015-10-22 06:30 +0200
    Re: [PATCH 1/8] mm: page_counter: let page_counter_try_charge()  return bool Michal Hocko <mhocko@kernel.org> - 2015-10-23 13:40 +0200
  [PATCH 6/8] mm: vmscan: simplify memcg vs. global shrinker invocation Johannes Weiner <hannes@cmpxchg.org> - 2015-10-22 06:30 +0200
    Re: [PATCH 6/8] mm: vmscan: simplify memcg vs. global shrinker  invocation Michal Hocko <mhocko@kernel.org> - 2015-10-23 15:30 +0200
  [PATCH 4/8] mm: memcontrol: prepare for unified hierarchy socket accounting Johannes Weiner <hannes@cmpxchg.org> - 2015-10-22 06:30 +0200
    Re: [PATCH 4/8] mm: memcontrol: prepare for unified hierarchy socket  accounting Michal Hocko <mhocko@kernel.org> - 2015-10-23 14:40 +0200
  [PATCH 5/8] mm: memcontrol: account socket memory on unified hierarchy Johannes Weiner <hannes@cmpxchg.org> - 2015-10-22 06:30 +0200
    Re: [PATCH 5/8] mm: memcontrol: account socket memory on unified  hierarchy Vladimir Davydov <vdavydov@virtuozzo.com> - 2015-10-22 20:50 +0200
    Re: [PATCH 5/8] mm: memcontrol: account socket memory on unified  hierarchy Michal Hocko <mhocko@kernel.org> - 2015-10-23 15:30 +0200
      Re: [PATCH 5/8] mm: memcontrol: account socket memory on unified  hierarchy David Miller <davem@davemloft.net> - 2015-10-23 15:50 +0200
        Re: [PATCH 5/8] mm: memcontrol: account socket memory on unified  hierarchy Johannes Weiner <hannes@cmpxchg.org> - 2015-10-26 18:00 +0100
          Re: [PATCH 5/8] mm: memcontrol: account socket memory on unified  hierarchy Michal Hocko <mhocko@kernel.org> - 2015-10-27 13:30 +0100
            Re: [PATCH 5/8] mm: memcontrol: account socket memory on unified  hierarchy David Miller <davem@davemloft.net> - 2015-10-27 14:40 +0100
            Re: [PATCH 5/8] mm: memcontrol: account socket memory on unified  hierarchy Johannes Weiner <hannes@cmpxchg.org> - 2015-10-27 16:50 +0100
              Re: [PATCH 5/8] mm: memcontrol: account socket memory on unified  hierarchy Michal Hocko <mhocko@kernel.org> - 2015-10-27 17:20 +0100
                Re: [PATCH 5/8] mm: memcontrol: account socket memory on unified  hierarchy Johannes Weiner <hannes@cmpxchg.org> - 2015-10-27 17:50 +0100
                Re: [PATCH 5/8] mm: memcontrol: account socket memory on unified  hierarchy David Miller <davem@davemloft.net> - 2015-10-28 01:30 +0100
                Re: [PATCH 5/8] mm: memcontrol: account socket memory on unified  hierarchy Johannes Weiner <hannes@cmpxchg.org> - 2015-10-28 04:10 +0100
                Re: [PATCH 5/8] mm: memcontrol: account socket memory on unified  hierarchy Michal Hocko <mhocko@kernel.org> - 2015-10-29 16:30 +0100
                Re: [PATCH 5/8] mm: memcontrol: account socket memory on unified  hierarchy Johannes Weiner <hannes@cmpxchg.org> - 2015-10-29 17:20 +0100
  [PATCH 2/8] mm: memcontrol: export root_mem_cgroup Johannes Weiner <hannes@cmpxchg.org> - 2015-10-22 06:30 +0200
    Re: [PATCH 2/8] mm: memcontrol: export root_mem_cgroup Michal Hocko <mhocko@kernel.org> - 2015-10-23 13:40 +0200
  [PATCH 7/8] mm: vmscan: report vmpressure at the level of reclaim activity Johannes Weiner <hannes@cmpxchg.org> - 2015-10-22 06:30 +0200
    Re: [PATCH 7/8] mm: vmscan: report vmpressure at the level of  reclaim activity Vladimir Davydov <vdavydov@virtuozzo.com> - 2015-10-22 20:50 +0200
    Re: [PATCH 7/8] mm: vmscan: report vmpressure at the level of  reclaim activity Michal Hocko <mhocko@kernel.org> - 2015-10-23 16:00 +0200
  [PATCH 8/8] mm: memcontrol: hook up vmpressure to socket pressure Johannes Weiner <hannes@cmpxchg.org> - 2015-10-22 06:30 +0200
    Re: [PATCH 8/8] mm: memcontrol: hook up vmpressure to socket pressure Vladimir Davydov <vdavydov@virtuozzo.com> - 2015-10-22 21:00 +0200
  [PATCH 3/8] net: consolidate memcg socket buffer tracking and accounting Johannes Weiner <hannes@cmpxchg.org> - 2015-10-22 06:30 +0200
    Re: [PATCH 3/8] net: consolidate memcg socket buffer tracking and  accounting Vladimir Davydov <vdavydov@virtuozzo.com> - 2015-10-22 20:50 +0200
      Re: [PATCH 3/8] net: consolidate memcg socket buffer tracking and  accounting Johannes Weiner <hannes@cmpxchg.org> - 2015-10-22 21:20 +0200
        Re: [PATCH 3/8] net: consolidate memcg socket buffer tracking and  accounting Vladimir Davydov <vdavydov@virtuozzo.com> - 2015-10-23 15:50 +0200
    Re: [PATCH 3/8] net: consolidate memcg socket buffer tracking and  accounting Michal Hocko <mhocko@kernel.org> - 2015-10-23 14:40 +0200
  Re: [PATCH 0/8] mm: memcontrol: account socket memory in unified  hierarchy Vladimir Davydov <vdavydov@virtuozzo.com> - 2015-10-22 20:50 +0200
    Re: [PATCH 0/8] mm: memcontrol: account socket memory in unified  hierarchy Johannes Weiner <hannes@cmpxchg.org> - 2015-10-26 18:30 +0100
      Re: [PATCH 0/8] mm: memcontrol: account socket memory in unified  hierarchy Vladimir Davydov <vdavydov@virtuozzo.com> - 2015-10-27 09:50 +0100
        Re: [PATCH 0/8] mm: memcontrol: account socket memory in unified  hierarchy Johannes Weiner <hannes@cmpxchg.org> - 2015-10-27 17:10 +0100
          Re: [PATCH 0/8] mm: memcontrol: account socket memory in unified  hierarchy Vladimir Davydov <vdavydov@virtuozzo.com> - 2015-10-28 09:30 +0100
            Re: [PATCH 0/8] mm: memcontrol: account socket memory in unified  hierarchy Johannes Weiner <hannes@cmpxchg.org> - 2015-10-28 20:00 +0100
              Re: [PATCH 0/8] mm: memcontrol: account socket memory in unified  hierarchy Vladimir Davydov <vdavydov@virtuozzo.com> - 2015-10-29 10:30 +0100
                Re: [PATCH 0/8] mm: memcontrol: account socket memory in unified  hierarchy Johannes Weiner <hannes@cmpxchg.org> - 2015-10-29 19:00 +0100

csiph-web