Path: csiph.com!usenet.pasdenom.info!dedibox.gegeweb.org!gegeweb.eu!nntpfeed.proxad.net!proxad.net!feeder1-1.proxad.net!hq-usenetpeers.eweka.nl!hq-usenetpeers.eweka.nl!bcyclone02.am1.xlned.com!bcyclone02.am1.xlned.com!newsfeed.xs4all.nl!newsfeed1.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
In-Reply-To: <542f8a27$0$12998$c3e8da3$5496439d@news.astraweb.com>
References: <CANm61jc=pB7n+5kuea6bmYXTbo3hXLT8d4JZx3Yq-TBtgGrFOQ@mail.gmail.com> <mailman.14590.1412378803.18130.python-list@python.org> <542f470b$0$13003$c3e8da3$5496439d@news.astraweb.com> <mailman.14591.1412389367.18130.python-list@python.org> <542f8a27$0$12998$c3e8da3$5496439d@news.astraweb.com>
Date: Sat, 4 Oct 2014 15:57:02 +1000
Subject: Re: "High water" Memory fragmentation still a thing?
From: Chris Angelico <rosuav@gmail.com>
Cc: "python-list@python.org" <python-list@python.org>
Content-Type: text/plain; charset=UTF-8
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.14594.1412402225.18130.python-list@python.org>
Lines: 33
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:78681

On Sat, Oct 4, 2014 at 3:48 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> I think that you're conflating a couple of different issues, although I
> welcome correction.
>
> I don't think that removing the GIL is a requirement for a memory compactor,
> or vice versa. I think that PyPy is capable of plugging in various
> different garbage collectors, including some without the GIL, which may or
> may not include memory compactors. So as far as I can tell, the two are
> independent.
>
> As far as the GIL in CPython goes, there have been at least two attempts to
> remove it, and they do show strong improvements for multi-threaded code
> running on multi-core machines. Alas, they also show significant *slowdown*
> for single-core machines, and very little improvement on dual-core
> machines.

Not conflating, comparing. In both cases, it's perfectly possible *in
theory* to build a CPython with this change. (GIL removal is clearly
possible in theory, as it's been done in practice.) And *in theory*,
there's some benefit to be gained by doing so. But it's not a case of
"why doesn't python-dev just knuckle down and do it already", as
there's no evidence that it's a real improvement. (A "compile to
machine code" feature might well be purely beneficial, and that's
simply a matter of work - how much work, for how many CPU
architectures, to get how much benefit. But at least that's going to
have a fairly good expectation of performance improvement.) A memory
compactor might well help a narrow set of Python programs (namely,
those which allocate heaps and heaps of memory, then throw away most
of it, and keep running for a long time), but the complexity cost will
make it unlikely to be of benefit.

ChrisA