Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #67955

Re: script uses up all memory

Show key headers only | View raw

On Fri, Mar 7, 2014 at 8:56 AM, Larry Martell <larry.martell@gmail.com> wrote:
> I figured out what is causing this. Each pass through the loop it does:
>
> self.tools = Tool.objects.filter(ip__isnull=False)
>
> And that is what is causing the memory consumption. If I move that
> outside the loop and just do that once the memory issue goes away. Now
> I need to figure out why this is happening and how to prevent it as
> they do want to query the db each pass through the loop in case it has
> been updated.

Interesting. So the next thing to do is to look into the
implementation of that. Does it allocate database resources and not
free them? Does it have internal reference loops?

Something to try: Put an explicit gc.collect() call into the loop. If
that solves your problem, you have a refloop somewhere (and you can
properly fix it by explicitly breaking the loop). If that keeps
returning large numbers, and especially if it populates gc.garbage
with a whole lot of stuff, then you definitely have refloops.

http://docs.python.org/2/library/gc.html

ChrisA

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

Re: script uses up all memory Chris Angelico <rosuav@gmail.com> - 2014-03-07 09:11 +1100

csiph-web