Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!xlned.com!feeder1.xlned.com!newsfeed.xs4all.nl!newsfeed3a.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path: <torriem+gmail@torriefamily.org>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.003
X-Spam-Evidence: '*H*': 0.99; '*S*': 0.00; 'python.': 0.02; 'essentially': 0.04; 'correct.': 0.07; 'indexing': 0.07; 'lookup': 0.09; 'true)': 0.09; 'dictionary,': 0.16; 'foo,': 0.16; 'from:addr:torriem': 0.16; 'from:name:michael torrie': 0.16; 'lookups.': 0.16; 'michael.': 0.16; 'numpy': 0.16; "object's": 0.16; 'retrieving': 0.16; 'slow,': 0.16; 'subject:slow': 0.16; 'variable.': 0.16; 'elements': 0.16; 'so.': 0.16; 'wrote:': 0.18; 'code,': 0.22; 'header:User-Agent:1': 0.23; 'specify': 0.24; 'typical': 0.24; 'file.': 0.24; 'header:In-Reply-To:1': 0.27; 'idea': 0.28; "doesn't": 0.30; '(like': 0.30; 'besides': 0.30; 'equivalent.': 0.31; 'faster,': 0.31; 'class': 0.32; 'probably': 0.32; 'run': 0.32; 'classes': 0.35; 'but': 0.35; 'there': 0.35; 'data,': 0.36; 'doing': 0.36; "didn't": 0.36; 'thanks': 0.36; "i'll": 0.36; 'too': 0.37; 'list': 0.37; 'message-id:@gmail.com': 0.38; 'to:addr:python-list': 0.38; 'list,': 0.38; 'pm,': 0.38; 'rather': 0.38; 'that,': 0.38; 'anything': 0.39; 'to:addr:python.org': 0.39; 'received:org': 0.40; 'how': 0.40; 'simply': 0.61; "you're": 0.61; 'first': 0.61; 'offer': 0.62; 'name': 0.63; 'kind': 0.63; 'different': 0.65; 'yes': 0.68; 'complexity': 0.84; 'imagine': 0.93
X-Virus-Scanned: amavisd-new at torriefamily.org
Date: Tue, 09 Sep 2014 09:32:19 -0600
From: Michael Torrie <torriem@gmail.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20131118 Thunderbird/17.0.11
MIME-Version: 1.0
To: python-list@python.org
Subject: Re: weakref, memory management and execution slow down in PyQt4
References: <540BFE43.5030006@riseup.net> <loom.20140907T152314-538@post.gmane.org> <540CADDD.3050501@riseup.net> <540CB15C.2060008@gmail.com> <540CC278.4050106@riseup.net> <540E40FD.8030706@gmail.com> <540E69AF.5040309@riseup.net>
In-Reply-To: <540E69AF.5040309@riseup.net>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.13904.1410276758.18130.python-list@python.org>
Lines: 33
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1410276758 news.xs4all.nl 2838 [2001:888:2000:d::a6]:36934
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:77740

Reposting to list, instead of directly to kjs

On 09/08/2014 08:45 PM, kjs wrote:
> Thanks for the consideration Michael. If you do get the data, and are
> able to run the code, let me know if you notice anything interesting.

Yeah I don't think I'll be able to have the time to download a 3 GB file.

>> Is there a reason you are using setattr and getattr instead of a proper
>> data structure?  both of those calls are rather expensive.  Would
>> probably be cheaper to use some kind of array, dictionary, or other
>> purpose-built data structure?
>>
>
> You're right, a dictionary can do everything I need and more. This
> happened to be the first thing I thought of, and I didn't imagine it
> would be very expensive. I figured it was simply a different way of
> defining and retrieving a class variable. IE setattr(self, foo, True) ==
> self.foo = True.

Yes you're correct.  It is the equivalent.  But it always involves
lookup in the object's dictionary, which is big O order O(n log n)
complexity for each and every access.  A list would be far faster,
essentially O(1) (I think?) after the single name lookup, since you can
access the elements by number.  Indexing into a list doesn't involve
doing name lookups; you just specify an offset.  No idea how much
faster, but significantly so.  If a list is too slow, there are other
array-like classes you can use (like numpy arrays) that do offer true
O(1) lookups.

Besides that, it's not typical use of setattr and getattr in python.