Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #78770

Re: "High water" Memory fragmentation still a thing?

Show key headers only | View raw

On Tue, Oct 7, 2014 at 3:21 AM,
<bryanjugglercryptographer@yahoo.com.dmarc.invalid> wrote:
> Chris Angelico wrote:
>> This is why a lot of long-duration processes are built to be restarted
>> periodically. It's not strictly necessary, but it can be the most
>> effective way of solving a problem. I tend to ignore that, though, and
>> let my processes just keep on running... for 88 wk 4d 23:56:27 so far,
>> on one of those processes. It's consuming less than half a gig of
>> virtual memory, quarter gig resident, and it's been doing a fair bit [...]
>
> A shipping product has to meet a higher standard. Planned process mortality is a reasonably simple strategy for building robust services from tools that have flaws in resource management. It assumes only that the operating system reliably reclaims resources from dead processes.
>

Sure, and that's all well and good. But what I just cited there *is* a
shipping product. That's a live server that runs a game that I'm admin
of. So it's possible to do without the resource safety net of periodic
restarts.

> For an example see the Apache HTTP daemon, particularly the classic pre-forking server. There's a configuration parameter, "MaxRequestsPerChild", that sets how many requests a process should answer before terminating.
>

That assumes that requests can be handled equally by any server
process - and more so, that there are such things as discrete
requests. That's true of HTTP, but not of everything. And even with
HTTP, if you do "long polls" [1] then clients might remain connected
for arbitrary lengths of time; either you have to cut them off when
you terminate the server process (hopefully that isn't too often, or
you lose the benefit of long polling), or you retain processes for
much longer.

Restarting isn't necessary. It's like rebooting a computer: people get
into the habit of doing it, because it "fixes problems", but all that
means is that it allows you to get sloppy with resource management.
Working under the constraint that your one process will remain running
for at least a year forces you to be careful, and IMO results in
better code overall.

ChrisA

[1] https://en.wikipedia.org/wiki/Push_technology#Long_polling

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Re: "High water" Memory fragmentation still a thing? Chris Angelico <rosuav@gmail.com> - 2014-10-04 09:26 +1000
  Re: "High water" Memory fragmentation still a thing? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-10-04 11:02 +1000
    Re: "High water" Memory fragmentation still a thing? Chris Angelico <rosuav@gmail.com> - 2014-10-04 12:22 +1000
      Re: "High water" Memory fragmentation still a thing? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-10-04 15:48 +1000
        Re: "High water" Memory fragmentation still a thing? Chris Angelico <rosuav@gmail.com> - 2014-10-04 15:57 +1000
        Re: "High water" Memory fragmentation still a thing? Sturla Molden <sturla.molden@gmail.com> - 2014-10-04 17:51 +0000
    Re: "High water" Memory fragmentation still a thing? Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-10-04 16:20 -0400
    Re: "High water" Memory fragmentation still a thing? mm0fmf <none@mailinator.com> - 2014-10-05 20:23 +0100
      Re: "High water" Memory fragmentation still a thing? Steven D'Aprano <steve@pearwood.info> - 2014-10-06 07:20 +0000
  Re: "High water" Memory fragmentation still a thing? bryanjugglercryptographer@yahoo.com - 2014-10-06 09:21 -0700
    Re: "High water" Memory fragmentation still a thing? Chris Angelico <rosuav@gmail.com> - 2014-10-07 03:43 +1100
      Re: "High water" Memory fragmentation still a thing? bryanjugglercryptographer@yahoo.com - 2014-10-08 10:28 -0700
        Re: "High water" Memory fragmentation still a thing? Emile van Sebille <emile@fenx.com> - 2014-10-09 14:39 -0700
        Re: "High water" Memory fragmentation still a thing? Chris Angelico <rosuav@gmail.com> - 2014-10-10 08:42 +1100
        Re: "High water" Memory fragmentation still a thing? dieter <dieter@handshake.de> - 2014-10-10 08:09 +0200
        Re: "High water" Memory fragmentation still a thing? Chris Angelico <rosuav@gmail.com> - 2014-10-10 17:52 +1100

csiph-web