Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!news.mixmin.net!feeds.phibee-telecom.net!newsfeed.xs4all.nl!newsfeed1.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
Date: Wed, 5 Mar 2014 17:27:31 -0500
Subject: script uses up all memory
From: Larry Martell <larry.martell@gmail.com>
To: "python-list@python.org" <python-list@python.org>
Content-Type: text/plain; charset=UTF-8
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.7845.1394058453.18130.python-list@python.org>
Lines: 59
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:67891

I have a script that forks off other processes and attempts to manage
them. Here is a stripped down version of the script:

        self.sleepTime = 300
        self.procs = {}
        self.startTimes = {}
        self.cmd = ['python', '/usr/local/motor/motor/app/some_other_script.py']

        while True:
            try:
                self.tools = Tool.objects.filter(ip__isnull=False)
            except Exception, e:
                print 'error from django call: ' + str(e)
                sys.exit(1)

            for tool in self.tools:
                name = tool.name
                if name in self.procs:
                    if self.procs[name].poll() is None:
                        if
(datetime.datetime.now()-self.startTimes[name]) >
datetime.timedelta(hours=12):
                            # it's been running too long - kill it
                            print 'killing script for ' + name + "
it's been running too long"
                            self.procs[name].kill()
                        else:
                            continue

                    if self.procs[name].returncode:
                        print 'scrikpt failed for ' + name + ', error
= ' + str(self.procs[name].returncode)

                    print 'starting script.py for ' + name + ' at ' +
str(datetime.datetime.now())
                    try:
                        self.procs[name] = subprocess.Popen(self.cmd)
                        self.startTimes[name] = datetime.datetime.now()
                    except Exception, e:
                        print 'error from Popen: ' + str(e)
                        sys.exit(1)
                else:
                    print 'starting script.py for ' + name + ' at ' +
str(datetime.datetime.now())
                    try:
                        self.procs[name] = subprocess.Popen(self.cmd)
                        self.startTimes[name] = datetime.datetime.now()
                    except Exception, e:
                        print 'error from Popen: ' + str(e)
                        sys.exit(1)

                time.sleep(self.sleepTime)


The script does what it's intended to do, however after about 2 hours
it has used up all the memory available and the machine hangs. Can
anyone see something that I am doing here that would be using memory
like this? Perhaps some immutable object needs to be repeatedly
recreated?