Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1499296

[RFC 0/6] Module for tracking/accounting shared memory buffers

Show key headers only | View raw

This patchstack introduces a new "memtrack" module for tracking and accounting
memory exported to userspace as shared buffers, like dma-buf fds or GEM handles.

Any process holding a reference to these buffers will keep the kernel from
reclaiming its backing pages.  mm counters don't provide a complete picture of
these allocations, since they only account for pages that are mapped into a
process's address space.  This problem is especially bad for systems like
Android that use dma-buf fds to share graphics and multimedia buffers between
processes: these allocations are often large, have complex sharing patterns,
and are rarely mapped into every process that holds a reference to them.

memtrack maintains a per-process list of shared buffer references, which is
exported to userspace as /proc/[pid]/memtrack.  Buffers can be optionally
"tagged" with a short string: for example, Android userspace would use this
tag to identify whether buffers were allocated on behalf of the camera stack,
GL, etc.  memtrack also exports the VMAs associated with these buffers so
that pages already included in the process's mm counters aren't double-counted.

Shared-buffer allocators can hook into memtrack by embedding
struct memtrack_buffer in their buffer metadata, calling
memtrack_buffer_{init,remove} at buffer allocation and free time, and
memtrack_buffer_{install,uninstall} when a userspace process takes or
drops a reference to the buffer.  For fd-backed buffers like dma-bufs, hooks in
fdtable.c and fork.c automatically notify memtrack when references are added or
removed from a process's fd table.

This patchstack adds memtrack hooks into dma-buf and ion.  If there's upstream
interest in memtrack, it can be extended to other memory allocators as well,
such as GEM implementations.

Greg Hackmann (1):
  drivers: staging: ion: add ION_IOC_TAG ioctl

Ruchi Kandoi (5):
  fs: add installed and uninstalled file_operations
  drivers: misc: add memtrack
  dma-buf: add memtrack support
  memtrack: Adds the accounting to keep track of all mmaped/unmapped
    pages.
  memtrack: Add memtrack accounting for forked processes.

 drivers/android/binder.c                |   4 +-
 drivers/dma-buf/dma-buf.c               |  37 +++
 drivers/misc/Kconfig                    |  16 +
 drivers/misc/Makefile                   |   1 +
 drivers/misc/memtrack.c                 | 516 ++++++++++++++++++++++++++++++++
 drivers/staging/android/ion/ion-ioctl.c |  17 ++
 drivers/staging/android/ion/ion.c       |  60 +++-
 drivers/staging/android/ion/ion_priv.h  |   2 +
 drivers/staging/android/uapi/ion.h      |  25 ++
 fs/file.c                               |  38 ++-
 fs/open.c                               |   2 +-
 fs/proc/base.c                          |   4 +
 include/linux/dma-buf.h                 |   5 +
 include/linux/fdtable.h                 |   4 +-
 include/linux/fs.h                      |   2 +
 include/linux/memtrack.h                | 130 ++++++++
 include/linux/mm.h                      |   3 +
 include/linux/sched.h                   |   3 +
 kernel/fork.c                           |  23 +-
 19 files changed, 875 insertions(+), 17 deletions(-)
 create mode 100644 drivers/misc/memtrack.c
 create mode 100644 include/linux/memtrack.h

-- 
2.8.0.rc3.226.g39d4020

Back to linux.kernel | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

[RFC 0/6] Module for tracking/accounting shared memory buffers Ruchi Kandoi <kandoiruchi@google.com> - 2016-10-12 02:00 +0200
  [RFC 1/6] fs: add installed and uninstalled file_operations Ruchi Kandoi <kandoiruchi@google.com> - 2016-10-12 02:00 +0200
  [RFC 4/6] memtrack: Adds the accounting to keep track of all mmaped/unmapped pages. Ruchi Kandoi <kandoiruchi@google.com> - 2016-10-12 02:00 +0200
  [RFC 5/6] memtrack: Add memtrack accounting for forked processes. Ruchi Kandoi <kandoiruchi@google.com> - 2016-10-12 02:00 +0200
  [RFC 6/6] drivers: staging: ion: add ION_IOC_TAG ioctl Ruchi Kandoi <kandoiruchi@google.com> - 2016-10-12 02:00 +0200
    Re: [RFC 6/6] drivers: staging: ion: add ION_IOC_TAG ioctl "Hillf Danton" <hillf.zj@alibaba-inc.com> - 2016-10-12 05:40 +0200
  [RFC 2/6] drivers: misc: add memtrack Ruchi Kandoi <kandoiruchi@google.com> - 2016-10-12 02:00 +0200
  Re: [RFC 0/6] Module for tracking/accounting shared memory buffers Al Viro <viro@ZenIV.linux.org.uk> - 2016-10-12 02:50 +0200
  Re: [RFC 0/6] Module for tracking/accounting shared memory buffers Rob Clark <robdclark@gmail.com> - 2016-10-12 03:20 +0200
  Re: [RFC 0/6] Module for tracking/accounting shared memory buffers Dave Hansen <dave.hansen@linux.intel.com> - 2016-10-12 19:30 +0200

csiph-web