Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #78660 > unrolled thread

"High water" Memory fragmentation still a thing?

Back to article view | Back to comp.lang.python

Contents

  "High water" Memory fragmentation still a thing? Croepha <croepha@gmail.com> - 2014-10-03 13:36 -0500

#78660 — "High water" Memory fragmentation still a thing?

[Multipart message — attachments visible in raw view] — view raw

Question:

Does python in general terms (apart from extensions or gc manipulation),
exhibit a "high water" type leak of allocated memory in recent python
versions (2.7+)?

Background:


>From the post:
http://chase-seibert.github.io/blog/2013/08/03/diagnosing-memory-leaks-python.html

Begin quote:

Long running Python jobs that consume a lot of memory while running may not
return that memory to the operating system until the process actually
terminates, even if everything is garbage collected properly. That was news
to me, but it's true. What this means is that processes that do need to use
a lot of memory will exhibit a "high water" behavior, where they remain
forever at the level of memory usage that they required at their peak.

Note: this behavior may be Linux specific; there are anecdotal reports that
Python on Windows does not have this problem.

This problem arises from the fact that the Python VM does its own internal
memory management. It's commonly know as memory fragmentation.
Unfortunately, there doesn't seem to be any fool-proof method of avoiding
it.

End Quote

However this paper seems to indicate that that is not a modern problem:
http://www.evanjones.ca/memoryallocator/


--Thanks
Dave Butler

[toc] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web