Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder1.news.weretis.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.xs4all.nl!newsfeed2a.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.003 X-Spam-Evidence: '*H*': 0.99; '*S*': 0.00; 'pointers': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'themselves,': 0.09; 'whatever.': 0.09; 'python': 0.11; 'jan': 0.12; 'accesses': 0.16; 'attributes,': 0.16; 'collections': 0.16; 'descriptor': 0.16; 'dictionaries': 0.16; 'massively': 0.16; 'operations?': 0.16; 'received:80.91.229.3': 0.16; 'received:plane.gmane.org': 0.16; 'reedy': 0.16; 'retrieving': 0.16; 'subclassing': 0.16; 'subject:dictionaries': 0.16; 'underlying': 0.16; 'userdict': 0.16; 'which,': 0.16; 'so.': 0.16; 'wrote:': 0.18; '3.0': 0.19; 'header:User-Agent:1': 0.23; '2.2': 0.24; 'proxy': 0.24; 'sort': 0.25; 'equivalent': 0.26; 'extension': 0.26; 'least': 0.26; 'subject:/': 0.26; 'values': 0.27; 'header:X-Complaints-To:1': 0.27; 'header:In-Reply-To:1': 0.27; 'am,': 0.29; 'properties': 0.29; "doesn't": 0.30; 'work.': 0.31; 'lists': 0.32; 'supposed': 0.32; '(i.e.': 0.33; 'cases': 0.33; 'objects': 0.35; 'subject:lists': 0.35; 'but': 0.35; 'there': 0.35; 'impression': 0.36; 'returning': 0.36; 'done': 0.36; 'doing': 0.36; 'method': 0.36; 'should': 0.36; 'easily': 0.37; 'to:addr:python-list': 0.38; 'rather': 0.38; 'does': 0.39; 'expensive': 0.39; 'to:addr:python.org': 0.39; 'received:org': 0.40; 'skip:u 10': 0.60; 'gone': 0.61; 'skip:o 30': 0.61; 'full': 0.61; 'simply': 0.61; 'simple': 0.61; 'kept': 0.65; 'direct': 0.67; 'accessed.': 0.84; 'dict()': 0.84; 'received:fios.verizon.net': 0.84; 'scratch,': 0.84; 'writing,': 0.84; 'received:108': 0.93 X-Injected-Via-Gmane: http://gmane.org/ To: python-list@python.org From: Terry Reedy Subject: Re: Lazy-evaluation lists/dictionaries Date: Sun, 26 Oct 2014 18:41:57 -0400 References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Gmane-NNTP-Posting-Host: pool-108-16-203-145.phlapa.fios.verizon.net User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 In-Reply-To: X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 30 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1414363336 news.xs4all.nl 2874 [2001:888:2000:d::a6]:58984 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:79732 On 10/26/2014 10:14 AM, Jon Ribbens wrote: > I have a need, in a Python C extension I am writing, for lists and > dictionaries with "lazy evaluation" - by which I mean that at least > some of the values in the lists/dictionaries are "proxy objects" > which, rather than returning as themselves, should return the thing > they are a proxy for when retrieved. This is because retrieving > the proxied objects is expensive and only a small minority of them > will actually be accessed, so retrieving them all before they are > actually accessed is massively inefficient. > > With object attributes, this can be easily done with the descriptor > protocol and having properties which look like simple objects but > actually cause a method to be invoked when they are accessed. However > there doesn't seem to be any equivalent way of doing this for > retrieving items from lists or dictionaries - unfortunately, the > method implementations of list() and dict() are full of direct > accesses to the underlying data pointers rather than indirecting > through obj->tp_as_mapping->mp_subscript or whatever. > > Is there any better way to do this other than simply re-implementing > these types from scratch, emulating all their methods and operations? > (i.e. using UserList/UserDict). I was under the impression that that > sort of thing was supposed to have gone since Python 2.2 or so. We considered dropping UserDict and UserList for 3.0 but kept them in collections for cases in which subclassing does not work. -- Terry Jan Reedy