Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.python > #77671

Re: weakref, memory management and execution slow down in PyQt4

Path csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!news.stack.nl!newsfeed.xs4all.nl!newsfeed4a.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path <python-python-list@m.gmane.org>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
DKIM-Signature v=1; a=rsa-sha256; c=relaxed/simple; d=python.org; s=200901; t=1410097318; bh=3wZyX9SBa42CuSh+TPcJJTpilzzTWHyNoKd9hLw+cQ4=; h=To:From:Subject:Date:References:From; b=nOsm88bXn+CuX+6Blq96btq7gilHwszH4FuQWTeMNSOqidM9zt1IFKjbtI7yAc79z y7Hiv5zTgZMf/ADD6NfEAm1wVfLlF33fT1w7n8TDkPcvEBt/BUF8ipfcj6QDOc8J25 U0t3Wcm4ZMRnCLAFRkf5cQVkDR46gcvE/wAodQBQ=
X-Spam-Status OK 0.008
X-Spam-Evidence '*H*': 0.98; '*S*': 0.00; 'causing': 0.04; 'tool,': 0.07; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'python': 0.11; 'gui': 0.12; 'suggest': 0.14; 'attributed': 0.16; 'finer': 0.16; 'from:name:antoine pitrou': 0.16; 'message-id:@post.gmane.org': 0.16; 'received:213.41.240': 0.16; 'received:213.41.240.54': 0.16; 'received:80.91.229.3': 0.16; 'received:charlus.yi.org': 0.16; 'received:plane.gmane.org': 0.16; 'received:yi.org': 0.16; 'stable.': 0.16; 'subject:slow': 0.16; 'from:addr:python.org': 0.16; "python's": 0.19; 'memory': 0.22; 'separate': 0.22; 'header :User-Agent:1': 0.23; 'replace': 0.24; 'visible': 0.24; 'regardless': 0.24; 'header:X-Complaints-To:1': 0.27; 'consumption': 0.31; 'further?': 0.31; 'writes:': 0.31; 'url:python': 0.33; 'older': 0.33; 'core': 0.34; 'objects': 0.35; 'but': 0.35; 'there': 0.35; 'next': 0.36; 'charset:us-ascii': 0.36; 'url:org': 0.36; 'application': 0.37; 'growing': 0.38; 'tools,': 0.38; 'url:library': 0.38; 'to:addr:python-list': 0.38; 'to:addr:python.org': 0.39; 'received:org': 0.40; 'how': 0.40; 'even': 0.60; 'remove': 0.60; 'analysis,': 0.60; 'analyze': 0.60; 'url:3': 0.61; 'first': 0.61; 'such': 0.63; 'believe': 0.68; 'analysis': 0.75; 'grow': 0.77; '3.4': 0.84; 'antoine.': 0.84; 'execution.': 0.84; 'isolate': 0.84; 'factors': 0.97
X-Injected-Via-Gmane http://gmane.org/
To python-list@python.org
From Antoine Pitrou <antoine@python.org>
Subject Re: weakref, memory management and execution slow down in PyQt4
Date Sun, 7 Sep 2014 13:28:56 +0000 (UTC)
References <540BFE43.5030006@riseup.net>
Mime-Version 1.0
Content-Type text/plain; charset=us-ascii
Content-Transfer-Encoding 7bit
X-Gmane-NNTP-Posting-Host sea.gmane.org
User-Agent Loom/3.14 (http://gmane.org/)
X-Loom-IP 213.41.240.54 (Mozilla/5.0 (X11; Linux x86_64; rv:32.0) Gecko/20100101 Firefox/32.0)
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.15
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.13848.1410097319.18130.python-list@python.org> (permalink)
Lines 29
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1410097319 news.xs4all.nl 2961 [2001:888:2000:d::a6]:57412
X-Complaints-To abuse@xs4all.nl
Xref csiph.com comp.lang.python:77671

Show key headers only | View raw


kjs <bfb <at> riseup.net> writes:
> 
> I have come to believe that the growing number of weakrefs is slowing
> down execution. Is my analysis misguided? How can I introspect further?
> If the slowdown can be attributed to weakref escalation, what are some
> next steps?

The way to analyze this is to build some gradually smaller subsets of your
application until you can isolate what is causing the growth in number of
objects (if any). I would suggest first remove the GUI and replace it with
some dummy functions, to stress your core logic.

Note that "top" isn't a very reliable tool, as memory fragmentation and
other factors can cause your process' visible size to grow even though
Python's memory consumption may be stable. There are dedicated Python tools
for finer analysis, such as tracemalloc, which is standard on 3.4 and available
as a backport for older versions:

https://docs.python.org/3/library/tracemalloc.html
http://pytracemalloc.readthedocs.org/

But regardless of such tools, the approach above (try to decompose your
workload into separate parts until your find the culprit) is highly recommended.

Regards

Antoine.

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread


Thread

Re: weakref, memory management and execution slow down in PyQt4 Antoine Pitrou <antoine@python.org> - 2014-09-07 13:28 +0000

csiph-web