Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #67955 > unrolled thread

Re: script uses up all memory

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: script uses up all memory Chris Angelico <rosuav@gmail.com> - 2014-03-07 09:11 +1100

#67955 — Re: script uses up all memory

On Fri, Mar 7, 2014 at 8:56 AM, Larry Martell <larry.martell@gmail.com> wrote:
> I figured out what is causing this. Each pass through the loop it does:
>
> self.tools = Tool.objects.filter(ip__isnull=False)
>
> And that is what is causing the memory consumption. If I move that
> outside the loop and just do that once the memory issue goes away. Now
> I need to figure out why this is happening and how to prevent it as
> they do want to query the db each pass through the loop in case it has
> been updated.

Interesting. So the next thing to do is to look into the
implementation of that. Does it allocate database resources and not
free them? Does it have internal reference loops?

Something to try: Put an explicit gc.collect() call into the loop. If
that solves your problem, you have a refloop somewhere (and you can
properly fix it by explicitly breaking the loop). If that keeps
returning large numbers, and especially if it populates gc.garbage
with a whole lot of stuff, then you definitely have refloops.

http://docs.python.org/2/library/gc.html

ChrisA

[toc] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web