Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python.announce > #3530 > unrolled thread

pymemtrace - Tracing Python Memory Usage

Back to article view | Back to comp.lang.python.announce

Contents

  pymemtrace - Tracing Python Memory Usage Paul Ross <apaulross@gmail.com> - 2020-11-17 16:31 +0000

#3530 — pymemtrace - Tracing Python Memory Usage

Welcome to *pymemtrace*, a collection of tools for monitoring and analysing
Python's memory usage including that of Python Extensions.
These tools work at various levels, at different granularities and with
different runtime costs so you can choose one appropriate to the problem at
hand.
They range from periodically measuring total memory usage down to reporting
every malloc and free call per Python line or C function call.

The tools are:

* *process* is a very lightweight way of logging the total memory usage at
regular time intervals.
  It can plot memory over time with plotting programs such as gnuplot.
  Some examples
https://pymemtrace.readthedocs.io/en/latest/examples/process.html
* *cPyMemTrace* is a memory tracer written in C that can report total
memory usage for every function call/return for
  both C and Python sections.
  Some examples:
https://pymemtrace.readthedocs.io/en/latest/examples/c_py_mem_trace.html
  and a technical note on cPyMemTrace:
https://pymemtrace.readthedocs.io/en/latest/tech_notes/cPyMemTrace.html.
* *DTrace*: There are a number of D scripts that can trace the low level
``malloc()`` and ``free()`` system calls and
  report how much memory was allocated and by whom.
  Some DTrace examples:
https://pymemtrace.readthedocs.io/en/latest/examples/dtrace.html
  and a technical note on DTrace
https://pymemtrace.readthedocs.io/en/latest/tech_notes/dtrace.html
* *trace_malloc* is a convenience wrapper around the Python standard
library tracemalloc module.
  This can take memory snapshots before and after code blocks and show the
change in memory caused by that code by module and line compensating for
the cost of tracemalloc itself.
  Some trace_malloc examples
https://pymemtrace.readthedocs.io/en/latest/examples/trace_malloc.html
* *debug_malloc_stats* is a wrapper around the sys._debugmallocstats
function that can take snapshots of
  memory before and after code execution and report the significant
differences in the Python small object allocator.
  Some debug_malloc_stats examples
https://pymemtrace.readthedocs.io/en/latest/examples/debug_malloc_stats.html

Within the documentation is also some (rather limited, for the moment)
advice on tracking down memory leaks with Python extensions.

On PyPi: https://pypi.org/project/pymemtrace/
Project: https://github.com/paulross/pymemtrace
Documentation: https://pymemtrace.readthedocs.io/en/latest/index.html

I hope that you find this useful as I do!

Paul Ross.

[toc] | [standalone]

Back to top | Article view | comp.lang.python.announce

csiph-web