Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #78681

Re: "High water" Memory fragmentation still a thing?

Show all headers | View raw

On Sat, Oct 4, 2014 at 3:48 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> I think that you're conflating a couple of different issues, although I
> welcome correction.
>
> I don't think that removing the GIL is a requirement for a memory compactor,
> or vice versa. I think that PyPy is capable of plugging in various
> different garbage collectors, including some without the GIL, which may or
> may not include memory compactors. So as far as I can tell, the two are
> independent.
>
> As far as the GIL in CPython goes, there have been at least two attempts to
> remove it, and they do show strong improvements for multi-threaded code
> running on multi-core machines. Alas, they also show significant *slowdown*
> for single-core machines, and very little improvement on dual-core
> machines.

Not conflating, comparing. In both cases, it's perfectly possible *in
theory* to build a CPython with this change. (GIL removal is clearly
possible in theory, as it's been done in practice.) And *in theory*,
there's some benefit to be gained by doing so. But it's not a case of
"why doesn't python-dev just knuckle down and do it already", as
there's no evidence that it's a real improvement. (A "compile to
machine code" feature might well be purely beneficial, and that's
simply a matter of work - how much work, for how many CPU
architectures, to get how much benefit. But at least that's going to
have a fairly good expectation of performance improvement.) A memory
compactor might well help a narrow set of Python programs (namely,
those which allocate heaps and heaps of memory, then throw away most
of it, and keep running for a long time), but the complexity cost will
make it unlikely to be of benefit.

ChrisA

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Re: "High water" Memory fragmentation still a thing? Chris Angelico <rosuav@gmail.com> - 2014-10-04 09:26 +1000
  Re: "High water" Memory fragmentation still a thing? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-10-04 11:02 +1000
    Re: "High water" Memory fragmentation still a thing? Chris Angelico <rosuav@gmail.com> - 2014-10-04 12:22 +1000
      Re: "High water" Memory fragmentation still a thing? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-10-04 15:48 +1000
        Re: "High water" Memory fragmentation still a thing? Chris Angelico <rosuav@gmail.com> - 2014-10-04 15:57 +1000
        Re: "High water" Memory fragmentation still a thing? Sturla Molden <sturla.molden@gmail.com> - 2014-10-04 17:51 +0000
    Re: "High water" Memory fragmentation still a thing? Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-10-04 16:20 -0400
    Re: "High water" Memory fragmentation still a thing? mm0fmf <none@mailinator.com> - 2014-10-05 20:23 +0100
      Re: "High water" Memory fragmentation still a thing? Steven D'Aprano <steve@pearwood.info> - 2014-10-06 07:20 +0000
  Re: "High water" Memory fragmentation still a thing? bryanjugglercryptographer@yahoo.com - 2014-10-06 09:21 -0700
    Re: "High water" Memory fragmentation still a thing? Chris Angelico <rosuav@gmail.com> - 2014-10-07 03:43 +1100
      Re: "High water" Memory fragmentation still a thing? bryanjugglercryptographer@yahoo.com - 2014-10-08 10:28 -0700
        Re: "High water" Memory fragmentation still a thing? Emile van Sebille <emile@fenx.com> - 2014-10-09 14:39 -0700
        Re: "High water" Memory fragmentation still a thing? Chris Angelico <rosuav@gmail.com> - 2014-10-10 08:42 +1100
        Re: "High water" Memory fragmentation still a thing? dieter <dieter@handshake.de> - 2014-10-10 08:09 +0200
        Re: "High water" Memory fragmentation still a thing? Chris Angelico <rosuav@gmail.com> - 2014-10-10 17:52 +1100

csiph-web